Creating Search Based Applications

A couple of months ago, I wrote a post on the “Big Picture” of Search. This post describes how the logical concept of SharePoint 2013 looks like, how the components interact with each other and what is the relationship between them.

This, more technical post has driven several people to ask me about the business background of Search Based Applications. How to start? What to do? What to expect? How to plan for a Search Based Application project?

Here are some tips and best practices of mine.

Search Based Applications (SBA)

First of all, let me clarify the definition of Search Based Applications. The main characteristics of SBAs are:

• The use of a search engine to dynamically drive the information processing interaction
• Triggering of information retrieval through key term or phrase query
• The use of search engine connectors to bring data from various sources
• Processing of user search terms in order to provide results more fulfilling the user’s intent
• Enhancements of the index
• Combining, ranking and formatting result sets
• Integrating information from various sources
• Performing document processing (previewing content, segmenting content, or recombining content)
• User interaction with content or data

SBAs usually fit into more characteristics of the above, if not all. In a nutshell: we use a search engine to collect the data from various systems (crawling & indexing). The User Interface gets the data not directly from a database or storage but from the Search index, where we store structured and unstructured data together, from various content sources, with various data schemas. The UI is responsible to present this wide variety of information to the users, meeting their intents, expectations and need, while supporting their everyday job as much as possible.

 

How to Start?

First of all, one has to identify the pain points to get rid of, the needs and requirements of the business to fulfill. Implementing a Search Based Application is a project where you have to know the goals. For example, initial requirements of a Customer Dashboard:

• Include information from various systems, like SharePoint, CRM, SAP, file shares, emailing, etc.
• Aggregate information related to a customer on a custom Dashboard page that can be reached from the Hover Panel.
• The dashboard has to contain the most important emails highlighted; the documents related to this customer, grouped by categories and business audiences (sales, development, project management, support, etc.); payment history on a diagram; office locations on a map; contact persons list; etc.
  The Hovel Panel contains a subset of this information, dynamically driven by the status of the customer (potential, new, closed, late in payment, etc.).
• Etc.

 

The Team

As every project, creating a Search Based Application needs a team as well. Of course, this means the group of people who work on the implementation, but also a lot of others. You need someone who is the owner of Search in your organization. Who doesn’t own the SBA project only, but who is responsible for the continuity of the Search service, who analyzes the reports, collects the suggestions, suggests improvements to the business stakeholders, etc. As I usually say, Search needs gardeners. Planting the nicest trees and flowers is not enough. You need to garden them to keep the beauty of your yard while it grows and changes continuously.

You also need to have a plan for this growth. This is where Search Governance comes into the picture.

 

UI Elements to Work With

Once you defined the goals, it’s time to think about the UI. What users are the target group of this application? How your users will get the most out of this application? What they want to see? How they do work? What information they need, on what views, in which order? Etc. Etc.

You have to know the elements of the Search Experience in SharePoint to be able to talk about it. Some examples, still for the Customer Dashboard:

• Search scenarios, the ways users will use the new Application. For example: Searching for customers in a specific location or area. Searching for customers with payment delay that is greater than 15 days. Opening the Customer Dashboard directly for customer with ID ABC123.
• Define the refiners to use, like location, industry, payment delay, products ordered, service needed, contact sales person, etc.
• On the Dashboard, display various Result Blocks for the different views of results. For example emails, documents, sales history, payment diagram, etc.
• Add custom data or a custom view to the Hover Panel. For example, instead of the default Created, Created By, etc. data you can display Customer ID, status, list of running projects, list of outstanding issues, etc.
• Add custom actions to the Hover Panel. For example, “See customer in CRM” or “See the orders in SAP” or “Send a payment deadline reminder”.

Besides these out-of-the-box elements you might need some custom components as well. For example, a map to display location information. Or a component for diagrams and charts. The limit is the sky…

Of course, it’s possible (and very likely) to define more than one search page in the application, in order to target the intent of users from different viewpoints. In this case, you have to identify the desired UI elements for each page. It’s very useful to create mockups for each and discuss them with the stakeholders.

 

Content Sources

Once you identified your goals and who will do what with the new application, it’s time to define the content sources that you want to work with now and later. Last summer, I wrote a post on Content Sources Best Practices, please use this for your reference here.

Besides crawling and indexing the various content sources, you have to be aware that Search Based Applications can use one or more Remote Search Indexes as well. For example if you would like to provide a “Related News” part on your Customer Dashboard, it might make sense to get the news from public sites like CNN or Financial Times. You might have customers who turns up in these news. The point here is to aggregate search-driven content into your application that you either don’t want or cannot crawl & index.

You can use a remote SharePoint (or FAST Search for SharePoint 2010) index this way, too. For example, if you have multiple farms in your enterprise.

 

Other considerations

Besides the special requirements of Search Based Applications, the implementation can be very similar to any common project. Deliverables make it to be special. If you prefer, it’s always good to follow an iterative, agile process where the business can follow the progress where everything is and can be really proactive. I really like this way of working.

It’s also important to talk about the performance. Usually, on the backend-side (crawling & indexing), a SBA application doesn’t need any more resources than any other search solution. These components don’t really care what we’ll do with the data stored in the index.
What might need more resources is the front-end: the WFE of SharePoint and the Query Processor. Maybe even the Analytics component.

 

Summary

What I would like to highlight in this post is three things: planning and gardening. Be prepared, plan your application, and don’t forget about the life “after”.