NFAIS: Emerging Technologies for Faster and More Effective Information Access

Del Sattertwhaite was up next talking about “Emerging Technologies for Faster and More Effective Information Access.” The focus of the talk was how do you get faster results from a software perspective to our searches. All of the improvements in search speed has been in response to hardware upgrades up till now.

Del gave us an example for how search engines work – how do you find the 6 smallest female salmon in a pond. Traditionally search engines grab all the fish. First they throw out the fish that are not salmon, then they throw out all of the male salmon, and finally they sort the females by size and pull the 6 smallest. The future model of search works like this: implant each fish with a chip with their vital stats and then radio the fish and have the smallest female salmon come to you.

Next we looked at federated search problems. Right now the issues include the fact that you’re checking multiple sources with multiple indexing methodologies. In the next gen model you pull content from all the sources so the query is consistent, apply a simple ranking algorithm for all the data and provide structured and unstructured queries from the same interface.

One real life example that Del gave us was the World Vital Records database. They had 12,000+ databases with 1.5 billion names. With the traditional approach a single query took 5 seconds but the index took 42 days to rebuild. With the new model it takes ~150 milliseconds to query and only 1.5 days to rebuild!

Freeing up this processing power will allow us to have power left over to increase understanding of search results and lower costs.

Technorati Tags:


  1. I’m confused about a couple of things, maybe you can clarify or maybe I need to go find Del.

    Do you happen to remember any other examples for the alternative to binary search of inverted indexes?

    And what does “pull content from the sources” mean in this context? Do you think it’s an indexing thing or happens at search time?



  2. Avi, I think you’ll need to ask Del, I was pretty much writing what he was saying when I could keep up.

Leave a Reply

Your email address will not be published. Required fields are marked *