Feeling the need for speed

As more agencies explore anew form of database technologycalled in-memory databases,contractors and other technologycompanies are stepping upwith solutions.In-memory databases promisefaster transaction speedsthan standard relational databasemanagement systems.Raytheon Co. of Lexington,Mass., is incorporating two suchdatabases for some of the shipboardelectronic and combatsystems of the Navy DDG 1000Zumwalt Class Destroyers,which the integrator is helpingNorthrop Grumman Corp.build.And last month, In-Q-Tel, theprivate venture capital firm createdby the CIA, made a strategicinvestment in StreamBaseSystems Inc., a Lexington,Mass.-based provider of in-memorydatabase software andassociated analysis tools.In-memory databases areoptimized for use in the workingmemory of machines. Usually, databases are stored inmain memory, typically on harddrives. When new material isgenerated, it is written to diskfirst, and when a query is madeof the database from a program,that material has to be read offthe disk.In contrast, in-memory databasesreside entirely in theworking memory, or RAM, of aserver or cluster of servers ?though they can be archived ondisk. Material is only written todisk later, if at all."The working dataset that theapplication will be using is resident,or persistent in memory,"said Patrick Moor, head of governmentcontracting and manufacturingfor Ants SoftwareInc. of Burlingame, Calif., oneof the companies chosen for theRaytheon work.The other company wasTimesTen, another inline databasecompany now owned byOracle Corp., of RedwoodShores, Calif.RAM works faster than harddrives, although it also is farmore expensive on a per-bytebasis. It also is volatile, meaningthe data is lost once thepower is shut off. But becausethese in-memorydatabases reside inRAM, they are generallyable to ingesthundreds of thousandsof transactions per second.They also can be queriedmore rapidly."For many applicationswhere you need to capture,react to and analyze that datainstantaneously, a database isjust too slow," said Bill Hobbib,vice president of marketing atStreamBase. With a traditionalrelational database managementsystem, "you are storingthe data before you query it. Wecan query the data at themoment it arrives."In tests, StreamBase hasshown that its software caningest as many as 500,000messages per second, whereasan RDBMS can, at most, take inabout 3,000 messages per second,Hobbib noted.For its destroyer work,Raytheon's Integrated DefenseSystems was looking for a databasethat could ingest a lot ofinformation from radar andsonar systems. Raytheon evengenerated an acronym todescribe the environment,CRUD ? create, replicate, updateand delete ? said PaulRivot, a director of competitivetechnologies at IBM, which issupporting Raytheon's work.Traditionally, to tackle theproblem, government contractorswould write a custom programthat would run an entiredatabase in working memory insuch a way that new materialwouldn't be written to disk first."It was so expensive, and youwould have to custom develop itfor each application," Rivotsaid. "Using an off-the-shelfmemory and database product,it is obviously cheaper."The Ants software combinesan in-memory database withregular RDBMS, so materialcan be stored through regularSQL commands. For its ownsoftware, StreamBase also keptclose with SQL as well. It keepsthe basic syntax, programmingprimitives and declarativenature of SQL. But theStreamBase software extendsthe language with additionalcapabilities, such as handlingdata that arrives out of sequentialorder, matching complexsequential patterns and detectingpatterns over periods oftime.A database programmercould learn StreamBase extensionsin about a day, Hobbibsaid. No standards body overseesthe company's extensions,but StreamBase is looking intothat possibility.In-memory databases are notthe only way to accelerate transactionand analysis speeds. Youcould also make the hard drivemuch faster.Texas Memory Systems Inc.,of Houston, offers hard storagesystems that appear to an operatingsystem as hard drives yetconsist entirely of much-fasterRAM units. When outfittedwith a standardRDBMS, such solidstatedrives couldprocess hundreds ofthousands of transactions persecond, said Woody Hutsell,executive vice president ofTexas Memory Systems. Eachunit can hold as much as 128G,and they can be tetheredtogether for more capacity.