In-Q-Tel embraces open-source search

Lucid Imagination will use an investment from the CIA's In-Q-Tel to develop products and the infrastructure to support the intelligence community's search needs.

The CIA’s investment arm, In-Q-Tel, announced an investment this week in Lucid Imagination, a new venture that seeks to do for open-source search platforms what Red Hat has done for the Linux operating system.

Lucid, of San Mateo, Calif., will use the funding to develop applications around Lucene and Solr and build the infrastructure to support customers in the intelligence community. The amount of funding from In-Q-Tel was not disclosed. Since September 2008, Lucid has raised $6.2 million in venture capital, including the money from In-Q-Tel.

Although Lucene and Solr have been around for a decade or more, the technologies have had little outside support, said Eric Gries, Lucid’s president and chief executive officer.

Users had to rely on the open-source community for support. “A lot of the implementations are now starting to be mission-critical, and users are looking for a commercial entity to stand behind the software,” Gries said.

Lucid was founded in late 2008 to take advantage of what Gries and others saw as pent-up demand for support services such as training, certified releases and quality assurance.

Without that kind of support, an open-source solution has trouble hitting the mainstream, said Donald Tighe, vice president of external affairs at In-Q-Tel.

“That is the opportunity we see here,” he said. “It makes us feel we have a long-term partner to develop and use the product.”

The arrangement fits with In-Q-Tel’s track record of investing in search technologies since the CIA founded the group in 1999, Tighe said. “This is a very real technology that the intelligence community has been aware of for a long time,” he added.

Lucene and Solr are full-text search technologies that offer several advantages, said Anil Uberoi, head of marketing at Lucid. The technologies are scalable, and one implementation of Lucene searches 6 billion documents, he added.

The involvement of the large open-source community ensures that innovations happen quickly and the technologies remain flexible.

“The ability to quickly build applications that meet specific purposes and have the full flexibility to make changes and modifications is very attractive,” Uberoi said.

In following the Red Hat model, Lucid plans to form partnerships with other large technology companies to use Lucene and Solr in their solutions.

“Stay tuned,” Uberoi said.