Sticky Glue

So the Big Question then arises – What makes FAST ESP different from perhaps SharePoint Search or Microsoft Search Server? There are really 6 Key Elements

1. Scalability
2. User Experience
3. Content Processing
4. Relevancy
5. Indexing
6. Linguistics

Let’s break them down, point by point:

1. Scalability

Scalability can really be defined as a 3D approach. FAST ESP can scale across Volume, Query Traffic and Index Latency.

Volume: I believe this is one of the key areas that anyone would possibly look to FAST ESP as an alternative. It can literary Span across Billions and Billions of documents and records. This on its own along with the speed it does it is a major draw card for looking at FAST ESP.

Query Traffic: This refers to the Queries per Second (QPS) that the search engine can process. FAST as an engine can do into the thousands of QPS’ and this can be scaled ,at a fairly large cost of hardware, to the millions. The Fact that I can grow it if I want to “staggering” levels, is a definite draw card.

Index Latency: This is simplified terms refers to how long I have to wait before new items/documents are available in the index. The extent that I can extend this with in FAST ESP allows me have “CACHED” indexes building dynamically at unheard of speeds. Where scalability is a definite draw card to FAST ESP, it is to be noted that this does come with a fairly high price on the hardware (to be discusses in a later post)

1. User Experience

   The flexibility of the interface is another major differentiator. As FAST ESP came from an Internet Based search engine, it has the flexibility to wrap neatly into your currently environment and sometimes Adding that slightly better user experience. As an example, take a look at www.gettyimages.com/catalyst/default.aspx (search for Africa and select the top option. Then drag the ideas in the tag cloud to the right and drop them on the empty box. Your pictures will change. Keep refining based upon the tag cloud suggestions and keep watching the pictures change) If you visit my session at Tech-Ed Africa, this year, FAST ESP – Architecture of Extreme Search, I show some other interesting interfaces imbedded directly into SharePoint.
   

2. Content Processing

   Now this is where FAST ESP really comes in on its own. FAST ESP can search through multiple type of content, from structured right through to completely unstructured. By that I mean FAST ESP can “Create” metadata on Content where there is no metadata. It has the “intelligence” to do what is called Entity Extraction, this is that it can look at a unstructured sentence and extract place names, people names And many other items like this. As an example login to www.ft.com (search for Microsoft, note the people, companies, topics, etc that we have pulled out of the content and allow you to refine your query on. Also, try clicking on the date viewer and see what a nice way of searching that is) So just from these 3 items so far, I’m sure you can start pinpointing the key differences so far? In the next post I’ll continue looking at Relevancy Indexing Linguistics
   

3. Structured Data Search
   

   FAST ESP gives you everything that you’ve come to be used to in search. Think of your standard search that you do on BING.COM, the way the result are return and the ability to link to metadata, such as meta tags.. etc.. So if you just use this point as the glue between standard search and FAST ESP.
   
   

4. Search Relevance
   

   FAST ESP gives you a number of ways to tweak and adjust the output of search results based on the user, user input, profile information and much more. In a later blog I’ll discuss some of the architectural components of FAST ESP that allows you pass a query or result through various pipelines and thus return a cross relevance of search results. If you are familiar with SharePoint Best Bets, then this will give you an idea of where FAST ESP scratches the surface.
   

5. Indexing
   

   The way FAST ESP does its indexing is possibly the one component that differentiates FAST from many other search technologies. When it comes to index latency, the time it takes from when a record / document changes to the point where the “item” is available for searching, FAST ESP can handle sub-second index retrieval as well as allowing the option of pushing items into the index. This gives FAST ESP a distinct advantage to have “live” results.
   

This gives you at least a high level idea as to how FAST ESP stands on its own when it comes to enterprise search. Over the next couple of weeks I’ll dive a bit deeper into the architecture of FAST ESP and hopefully soon I’ll have an opportunity to look into the FAST for SharePoint in SharePoint 2010.