Start your engines

The General Services Administration's recompete for the FirstGov search engine contract shows the increased sophistication that government needs from its query tools.

Today, when someone wants to find information through FirstGov, the one-stop portal for government information, the search function will return mostly Web pages. But GSA is mandating that FirstGov's next iteration help users find information in other online public documents as well, such as Word documents, Adobe Acrobat files, database records and many other file formats, according to the Jan. 7 request for proposals.

In addition to this increased versatility, the search engine must also be hardy enough to index up to 200 million pages, field requests from 17 million visitors per year and return multiple results ranked in terms of relevance.

"The new RFP definitely pushes the requirements from what they were," said Chris Sherman, president of search engine analysis company Searchwise Inc., Boulder, Colo. "I don't believe any one vendor can satisfy all the particular requirements."

Launched in September 2000 by GSA, FirstGov is linked to 22,000 federal Web sites. The search engine that now services the site was provided by Inktomi Corp., Foster City, Calif., through the nonprofit Federal Search Foundation Inc., Washington.

As government organizations become more intranet and Internet-dependent, they'll rely more on search engines to help employees find the information they need to do their jobs.

The worldwide search tool market for 2001, which includes commercial and government Web site and intranet searches, was approximately $1.2 billion in 2001, and will grow to $2.5 billion by 2006, according to estimates by analyst and consulting firm Ovum, Wakefield, Mass.

To better equip the workplace, search vendors are responding with next-generation features to differentiate themselves, such as security enhancements, knowledge management tools, multiple format searching and ? that most elusive feature of all ? more relevant results to queries.

"You can get search engines off the Internet for free," said Anthony Bettencourt, president of enterprise search engine provider Verity Inc., Sunnyvale, Calif., explaining why his company added new security features to its enterprise search solution. "This was a way to give ourselves some insulation and keep our price points healthy."

"Next-generation search is not one technology, but rather a broad range of innovations, rediscoveries and adaptations aimed at making search a much more relevant and dependable process and, therefore, more valuable," according to a July 2001 Ovum report.

Smartlogik Inc., Alexandria, Va., has developed a knowledge management system where internal documents are automatically cate-
gorized in a librarylike classification system, called a taxonomy, which the company claims produces more relevant returns.

The company has also developed something it calls the "probabilistic model" of searching, where the more unique a term is in the user's search, the more likely it is to be the key element of the search. So documents with prominent use of that term will be at the top of the list.

"We see ourselves as the next generation of intelligent searching," said Ray Fruglia, vice president of marketing.

Web-site search provider Atomz Corp., San Francisco, has paired its search offering with a content management system, allowing organizations to format Web documents and instantly index them for searches, said Steve Kusmer, chief executive officer of Atomz.

The company has also established an application-service-provider type of service, where organizations, such as the U.S. Customs Service, outsource the of their search page entirely to a Atomz.

Zylab International Inc., Germantown, Md., a longtime government supplier of text search services, was purchased in January by AuthentiDate Holding Corp., Schenectady, N.Y., an authentication services provider.

The purchase allows AuthentiDate to offer a search function that can guarantee legal validity of an electronic or paper document's date of origination, which the company plans to market to courts and law firms.

According to vice president Phil Martin, the combined technologies of the two companies could produce a better return on investment than what Zylab could achieve with its own search products.

The $8 million, five-year FirstGov project has attracted more than 80 bidders, including traditional Web search engine vendors such as Alta Vista Co., Palo Alto, Calif., as well as vendors of enterprise information management tools, such as SmartLogik.

Inktomi plans to re-bid that contract not with an update of the Web-based search tool now in place, but rather with its enterprise search solution, said Andrew Littlefield, Inktomi's chief strategist and director of product management. That solution, the company believes, will be better suited to indexing the wide variety of formats GSA requires.

Another competitor is Chicago-based Divine Inc., an integrated solutions provider that recently purchased the Web search engine provider Northern Light Technology LLC. According to David Seuss, ex-chief executive officer of Northern Light and now general manager for Divine's search and content division, Divine's competitive edge will be its software's ability to index large numbers of documents.

Starting out in 1997 as a search company that canvassed the entire Web, Northern Light has cataloged more than 400 million Web pages. Divine's robust Web-crawling technology easily meets FirstGov's performance requirements, Seuss said.

"Scale is an important issue. There are not many companies that can search to 20 million queries a year," Seuss said.

In January, Northern Light discontinued its public search engine, which was suffering from flat banner ad sales. According to Seuss, the company is now marketing its technology for the enterprises needing to quickly locate information on the Web about competitors or other parties of interest.

Also in January, Northern Light was chosen by In-Q-Tel, the research investment arm of the Central Intelligence Agency, to help the CIA search the Web in multiple languages for information that would be of interest to the agency.

In contrast, the internal enterprise searching has traditionally placed more value on software that can work with a wide variety of file formats, such as spreadsheets, databases and e-mail.

This was where Verity made its mark. Formed in 1988, the enterprise search solution provider boasts more than 1,500 corporate customers, including government clients such as the General Services Administration and the state of California, which uses its search solution for the MyCalifornia portal.

Although offering searches in multiple formats was Verity's first calling card, Bettencourt said the company developed a whole range of new services. One is that Verity's engine will verify that an employee can see a particular document before it is even presented.

"If you slip up and give a rogue user the ability to see a file called 'layoffs,' regardless if they can access it, you're in trouble," Bettencourt said. "We've built a system that looks up all the permissions that users of that documents are able to access. If a document shows up that they aren't allowed to access, we flush it before it is viewed."

Staff Writer Joab Jackson can be reached at jjackson@postnewsweektech.com.