Office 365 Search

The Future of SharePoint – Search and Discovery

The Future of SharePoint – Search and Discovery

Today, at the Future of SharePoint event in San Francisco, Microsoft unveils the new cloud-first, mobile-first vision and roadmap for SharePoint, along with new innovations that empower people and organizations to intelligently discover, share and collaborate.

Unfortunately, I cannot attend the event in person, but I’m sharing my thoughts in this post.

The vision for unified access and intelligent discovery are very exciting . On modern intranets, “findability” takes place of “search“, “discovery” becomes more and more valued in the age of information overload.

read more…

Office Graph and Delve news

Office Graph and Delve news

I have been quite off-grid recently, but life goes on and as always, there are so many news about Office 365. Some of them are not related to search and findability at all (or at least, not directly), but some others are quite interesting.

In this post, I’ve collected the most important and (hopefully) exciting ones. read more…

Delve Days of December: Free Expert Webinars for Delve & Office Graph

Delve and Office Graph are appearing in Office 365 tenancies around the globe. These truly innovative technologies combine search, big data, and enterprise social networking to change the story about how users find and discover information and each other. And, in 2015, the power of Delve and Office Graph will extend into hybrid, on-premises environments as well!

IT Unity is stepping up to help the SharePoint & Office 365 communities succeed with Delve & Office Graph. We’re providing a series of FREE WEBINARS designed to give you everything you need to understand, plan for, implement, and develop for two of the most potentially revolutionary services Microsoft has released: read more…

My First Time with Delve

When I saw Delve (that time Codename: Oslo) at SPC in Las Vegas, it was an instant love at first sight. After this first meeting, I had to wait for a while until I could really “touch” it.

The first time when it got enabled on a tenant I use, was in a customer’s environment. I do have an account to this customer, for several reasons. First time when I saw Delve was there, I had an immediate curiosity—I’ve never looked around at that environment before to check what content is shared with me. I worked with documents related to my project but that’s all.

Once I opened, Delve presented LOTS of documents which were shared with me. Much more than I expected!

The top ones were the customer’s roadmap and research plans! Even their titles told me a lot about their plans and how they want to move forward on the market. Even without opening these documents I’ve got more info on their plans than ever before. I work remotely there, most of our meetings are well-targeted, time-limited online calls, we don’t have time and opportunity for “soft” talking.

But these new “findings,” presented by Delve were good reasons to initiate new discussions with this customer. And guess what?—They really appreciated my thoughts and feedback. These discussions are still ongoing, and very useful for both of us! I can learn a lot from and about this customer of mine, and they can get the benefits of my “external” thoughts. And this way, our relationship got stronger now than ever before.

I’m pretty confident, these discussions might have never happened without Delve…

A Business Overview of Office Graph and Office Delve – on IT Unity

Office Delve and the Office Graph are officially released and are being deployed into Office 365 tenancies worldwide. So it’s time to catch up with what these technologies do, what they mean to business, and what they reveal about the new Microsoft. – Recently, Dan Holme and I published the first part of a series on IT Unity, in which we discuss the basics and most important benefits of these technologies innovations. read more…

Delve Is Missing On My Tenant. What Should I Do?

I get more and more questions like, “Delve is missing on my tenant. What should I do?”

In this article, I’d like to summarize the steps needed to “activate” Office Graph and Delve in your tenant and some troubleshooting steps if it’s not enough.

First of all, to activate Office Graph and Delve, please find the required steps here: https://support.office.com/en-us/article/Office-Delve-for-Office-365-admins-54f87a42-15a4-44b4-9df0-d36287d9531b

Please note that after enrolling to the First Release program, it might take up to 24 hours to get the new features like Office Graph and Delve.

If after 24 hours you still cannot see “Delve” on the navigation bar (like me), here are some ideas to try.

    1. Try to run a GQL query against Office Graph in your tenant, to prove Office Graph works. For example: https://[yourtenant]/_api/search/query?QueryText=’*’&Properties=’GraphQuery:actor(me\,action\:1021)

If Office Graph works, you’ll get ten documents authored by you recently.

  1. If Office Graph works, try to see if Delve works by opening this URL: https://[your-mysite-tenant]/_layouts/15/me.aspx?v=home

If you get documents here, Delve works fine; it just doesn’t get displayed on your navigation. It worth a try to check “My Apps;” Delve might be hidden there.

(On my tenant, this is the case – both Office Graph and Delve work fine but Delve does not display in the navigation.)

In any case: if you have issues on any level, open a support ticket and explain your issue with as many details as possible. The Product Group is already investigating in it, but the more tenants they know with these issues, the easier troubleshooting gets.

Privacy in Delve and Office Graph

As Delve is getting into more and more tenants, people obviously talk about it more and more. One can find blog posts and articles here and there, with excitement as well as with skepticism—like any innovation.

While reading the online content about Office Graph and Delve, one of the biggest concerns I’ve met is privacy. For example:

  • I’m concerned that subcontractors and other external partners of my company will be able to discover and see content that was not intended to be shared with them (see my “First time with Delve” story).
  • Let’s say I’m planning to apply for a new position opening in the company, but I don’t want my current manager to know about it yet. I have already checked the related HR documents (job description, apply criteria, rules and policies, etc.). Office Graph stores all of my activities, including that I viewed these documents. What if my manager checks it and figures out I’m planning to leave his team?
  • Let’s say, as a woman, I get pregnant, but I don’t want to share this information too early. But, I want to know my rights, the benefits I can get from the company, etc., without “releasing” the information of being pregnant.
  • Office Graph “knows” everything about me: what content I view, what meetings I have, who I send emails to and get emails from, which documents are presented to me, etc. How does Office Graph handle the sensitive content? What if I don’t want to share what people I communicate and collaborate with and what content I work with?

Let me clarify how privacy works in Office Graph and Delve, and what kind of relationships can be “surfaced” and how.

Sensitive content

Office Graph, and therefore its presentation layer, Delve, always provides security-trimmed results, similar to the “traditional” Search experience in SharePoint and Office 365. If you don’t have permissions to see a document, it won’t be presented to you at all. If you do have permissions to it, you can find it.

Office Graph does the security trimming in the backend; there is no way to “hack” or work around it. Delve cannot be a “security leak” in your organization.

For example: Let’s say you attend a strategic meeting with other C-level managers of the company, and there’re some content presented there. If the permission settings on these documents are set correctly and only the proper managers can see it, you don’t have to worry about Delve at all. People who have no access to the documents cannot see this in Delve at all. These documents won’t be presented to them in any way, because Office Graph does security trimming before it sends the content to Delve.

Conclusion: if some content is sensitive AND its permissions are set correctly, people cannot “discover” it by accident.

Sensitive “relationships”

In some cases, the content itself is public, but we don’t want to let others know we view it. For example, when I want to apply for a new position in the company. Or when a woman gets pregnant.

Office Graph fully respects privacy. To explain how it does, I have to explain how Office Graph works and how it stores all the information. Office Graph is a real, mathematical graph. It has nodes and edges. It’s nodes are the users and the content, edges the relationships. The edges always point from the Actor of the relationship (user) to the object of the relationship (document or another user).

Each edge (relationship) has several characteristics, like type or weight (importance). One of these characteristics is the visibility information. An edge (relationship) can be public or private:

  • Public: A public relationship is visible for everyone. An object that is connected to the Actor by a public edge can be discovered by others using Delve (with security trimming, of course!) For example, “modify” or “create” relationships are always public; people can discover content modified or created by someone else as Actor.
  • Private: An object, which is connected to the Actor by a private edge, cannot be discovered by other users. For example, an edge that represents a “view” relationship is always private. If I view a document, nobody will know about it.

For Example: When I open the HR documents for a new position opening, Office Graph stores an edge (relationship) that points from me (Actor) to the HR document (Object). If I only view (read) the document, this relationship will be private — nobody can discover the content by its relationship to me.

Conclusion: If someone opens documents for view only, the relationship gets stored into the Office Graph as a private edge. This means I get to have a connection to these documents but nobody else will be able to surface it. I am safe, even if I am in the first period of my pregnancy or planning to leave the company.

Summary

The innovation of Office Graph and Delve raises up several questions. One of the most important concern is privacy — how these new tools support and respect privacy. This concern is reasonable, of course: nobody wants others to discover information that they’re not supposed to discover.

For permission management, the same mechanism is used in Office Graph as in Search: everything is security trimmed on the engine level; nothing can be surfaced to the user interface. There is no way to work around this.

Moreover, in Office Graph we can also find two different edges: public edges allow us to discover content, but private edges don’t. Sensitive relationships cannot get discovered at all.

Creating Search Based Applications

A couple of months ago, I wrote a post on the “Big Picture” of Search. This post describes how the logical concept of SharePoint 2013 looks like, how the components interact with each other and what is the relationship between them.

This, more technical post has driven several people to ask me about the business background of Search Based Applications. How to start? What to do? What to expect? How to plan for a Search Based Application project?

Here are some tips and best practices of mine.

Search Based Applications (SBA)

First of all, let me clarify the definition of Search Based Applications. The main characteristics of SBAs are:

  • The use of a search engine to dynamically drive the information processing interaction
  • Triggering of information retrieval through key term or phrase query
  • The use of search engine connectors to bring data from various sources
  • Processing of user search terms in order to provide results more fulfilling the user’s intent
  • Enhancements of the index
  • Combining, ranking and formatting result sets
  • Integrating information from various sources
  • Performing document processing (previewing content, segmenting content, or recombining content)
  • User interaction with content or data

SBAs usually fit into more characteristics of the above, if not all. In a nutshell: we use a search engine to collect the data from various systems (crawling & indexing). The User Interface gets the data not directly from a database or storage but from the Search index, where we store structured and unstructured data together, from various content sources, with various data schemas. The UI is responsible to present this wide variety of information to the users, meeting their intents, expectations and need, while supporting their everyday job as much as possible.

 

How to Start?

First of all, one has to identify the pain points to get rid of, the needs and requirements of the business to fulfill. Implementing a Search Based Application is a project where you have to know the goals. For example, initial requirements of a Customer Dashboard:

  • Include information from various systems, like SharePoint, CRM, SAP, file shares, emailing, etc.
  • Aggregate information related to a customer on a custom Dashboard page that can be reached from the Hover Panel.
  • The dashboard has to contain the most important emails highlighted; the documents related to this customer, grouped by categories and business audiences (sales, development, project management, support, etc.); payment history on a diagram; office locations on a map; contact persons list; etc.
    The Hovel Panel contains a subset of this information, dynamically driven by the status of the customer (potential, new, closed, late in payment, etc.).
  • Etc.

 

The Team

As every project, creating a Search Based Application needs a team as well. Of course, this means the group of people who work on the implementation, but also a lot of others. You need someone who is the owner of Search in your organization. Who doesn’t own the SBA project only, but who is responsible for the continuity of the Search service, who analyzes the reports, collects the suggestions, suggests improvements to the business stakeholders, etc. As I usually say, Search needs gardeners. Planting the nicest trees and flowers is not enough. You need to garden them to keep the beauty of your yard while it grows and changes continuously.

You also need to have a plan for this growth. This is where Search Governance comes into the picture.

 

UI Elements to Work With

Once you defined the goals, it’s time to think about the UI. What users are the target group of this application? How your users will get the most out of this application? What they want to see? How they do work? What information they need, on what views, in which order? Etc. Etc.

You have to know the elements of the Search Experience in SharePoint to be able to talk about it. Some examples, still for the Customer Dashboard:

  • Search scenarios, the ways users will use the new Application. For example: Searching for customers in a specific location or area. Searching for customers with payment delay that is greater than 15 days. Opening the Customer Dashboard directly for customer with ID ABC123.
  • Define the refiners to use, like location, industry, payment delay, products ordered, service needed, contact sales person, etc.
  • On the Dashboard, display various Result Blocks for the different views of results. For example emails, documents, sales history, payment diagram, etc.
  • Add custom data or a custom view to the Hover Panel. For example, instead of the default Created, Created By, etc. data you can display Customer ID, status, list of running projects, list of outstanding issues, etc.
  • Add custom actions to the Hover Panel. For example, “See customer in CRM” or “See the orders in SAP” or “Send a payment deadline reminder”.

Besides these out-of-the-box elements you might need some custom components as well. For example, a map to display location information. Or a component for diagrams and charts. The limit is the sky…

Of course, it’s possible (and very likely) to define more than one search page in the application, in order to target the intent of users from different viewpoints. In this case, you have to identify the desired UI elements for each page. It’s very useful to create mockups for each and discuss them with the stakeholders.

 

Content Sources

Once you identified your goals and who will do what with the new application, it’s time to define the content sources that you want to work with now and later. Last summer, I wrote a post on Content Sources Best Practices, please use this for your reference here.

Besides crawling and indexing the various content sources, you have to be aware that Search Based Applications can use one or more Remote Search Indexes as well. For example if you would like to provide a “Related News” part on your Customer Dashboard, it might make sense to get the news from public sites like CNN or Financial Times. You might have customers who turns up in these news. The point here is to aggregate search-driven content into your application that you either don’t want or cannot crawl & index.

You can use a remote SharePoint (or FAST Search for SharePoint 2010) index this way, too. For example, if you have multiple farms in your enterprise.

 

Other considerations

Besides the special requirements of Search Based Applications, the implementation can be very similar to any common project. Deliverables make it to be special. If you prefer, it’s always good to follow an iterative, agile process where the business can follow the progress where everything is and can be really proactive. I really like this way of working.

It’s also important to talk about the performance. Usually, on the backend-side (crawling & indexing), a SBA application doesn’t need any more resources than any other search solution. These components don’t really care what we’ll do with the data stored in the index.
What might need more resources is the front-end: the WFE of SharePoint and the Query Processor. Maybe even the Analytics component.

 

Summary

What I would like to highlight in this post is three things: planning and gardening. Be prepared, plan your application, and don’t forget about the life “after”.

Refiners in SharePoint 2013 Search

The concept of refiners is not new, it’s been in SharePoint since the version 2010. But there was a big difference between SharePoint 2010’s refiners and FAST Search for SharePoint’s refiners: while we had shallow refiners in SharePoint 2010 (SP2010), we had deep refiners with item counts in FAST Search for SharePoint (FS4SP). I’ve made a comparison on them earlier, please find my related blog post here.

As you’re already aware of, there’s only one, integrated Search core in SharePoint 2013 (see more details in our e-book here).

After this short introduction and historical info, you might expect we have deep refiners in SharePoint 2013, right? It’s so logical. But when you open a SP2013 Search Center, you find something like this:

It’s nice we have some visual refiner in the bottom, but why we don’t have item counts for the text refiners? Are they shallow refiners? If they’re shallow, how come the counts can be represented on the visual diagram-like refiner?

The truth is the following: we do have deep refiners in SharePoint 2013 (this is why the counts can be represented on the Modified Date refiner above), but by default the counts are hidden for text refiners.

In order to display the counts, please let me introduce you the new Refiner Panel.

Refinement Panel is still a web part, but very different than it was in the previous versions. When you open its properties, you’ll see something like this:

Although there are some very exciting options here, please let me focus on the button “Choose Refiners” for now. Once you click on it, a nice and fancy wizard will be displayed:

 

Let’s see what we have here.

First, there’s a visual control for selecting the refiners we need on our Search page. Wow, no more Config XML magic! Cool!

On the left side there’s the list of available refiners (managed properties), on the right side you can see your current selections. Of course, you can add and remove as well as reorder the refiners here.

When you select a refiner in the list above, you’ll see its configuration in the bottom half of the screen. Things you can configure here:

  • Display name
  • Display template (see below)
  • Sort by
  • Sort direction (ASC / DESC)
  • Max number of refiner values

As MSDN says about Display Templates:

“You may want to control the format and presentation of search results on your website. You can do that with display templates, which extend the options available for customizing search results through the user interface beyond mapping the predefined fields that you want to display.

You can use display templates when you want to customize how content displays on your site in a way that goes beyond mapping predefined fields. Display templates are part of the Designer Manager workflow in SharePoint 2013. There are three contexts when you may want to use display templates with search results—when you want to map how the overall structure of search results are presented, when you want to show groups of results, and when you want to show how each result, or item, in the result set is displayed. These are called Control, Group, and Item templates, respectively.”(See more here.)

So that, in case of refiners, the selected Display Template defines how to display the given refiner. If you open the dropdown, you can see and select the existing templates – out of the box, you have the followings:

  • Refiner Item
  • Multi-value Refiner Item

If you want to display the item counts, there are two options:

  1. You can modify the existing “Refiner Item” display template, OR
  2. You can create a custom display template.

Let me choose the 2nd option and create my own display template, with the name “Refiner Item with Item Counts”.

To do this, you can go to the Design Manager and create a new one, or you can take the existing “Refiner Item” (stored as the Master Page gallery / Display Templates / Filters / Filter_Default.html). You can either modify it (if you want to modify the behavior of the default one) – or create a new one (cop this one and modify what you want to get changed).

There are two things to modify here in order to get the item counts:

  • Give a new Title to it, something like this:

    <title>Refinement Item with Count</title>

  • Look for the value “ShowCounts”. By default, it’s set to “false”, this is why the item counts are hidden. Change it to “true”:

    this.Options = {

    ShowClientPeoplePicker: false,

    ShowCounts: true

    };

Save your file and upload to the Master Page Gallery.

Note, there’s a .js file for each other design templates. Don’t worry about it, this will be created when you upload the HTML.

That’s it. If you go back to the Refiner Panel’s properties and open the Display Template dropdown, you’ll see the option “Refiner Item with Count”. Select this for each refiner value you would like to see the counts – that’s it!

You’ll see the deep refiners with item counts, in the very same way as you did in FAST Search for SharePoint 2010. And don’t forget: you can do this even on Office 365!