Legacy systems get smart

Military commanders want to see where their troops and equipment are at all times. Tax bureaus want to flag suspicious returns when they are filed. The Homeland Security Department wants to assemble data from numerous sources to pinpoint where terrorists might strike. In many instances, however, the data they need to gather and assemble for this sort of high-level, instant visibility often lives in legacy systems. Sensing opportunity, providers of middleware -- those tools that extract and combine information from multiple legacy systems -- are adding real-time analytics to their offerings. The need is clear: While real-time analytics is often a feature that comes with new database systems, many agencies don't have the time or money to buy new systems.The Homeland Security Department, for instance, will integrate existing systems rather than build new ones wherever feasible, said S.W. "Woody" Hall, assistant commissioner for information and technology, at recent conference. Yet, the analysis DHS needs must be delivered as quickly as possible to avoid another catastrophe such as the Sept. 11 terrorist attacks. The buzzword in Defense Department logistics offices these days is visibility. Commanders want to see what resources they have on hand at that exact moment. For integrators such as Stanley Associates Inc., Alexandria, Va., supplying that real-time information can be a challenge, given that many of the sources they work with are legacy systems, such as IBM OS/390 Cobol-driven mainframes or DOS-based personal computers. "We spend a lot of time extracting data from various sources. If there were a capability that would allow me to run the same reports in real-time without having to transform and stage the data, that would be a good thing," said Jim Brabston, Stanley Associates' vice president of defense systems. Vendors are taking note. In May, middleware provider WebMethods Inc., Fairfax, Va., partnered with business intelligence provider Informatica Corp., Redwood City, Calif., to provide a solution the two are marketing jointly, called the Business Activity Platform. WebMethods provides the software that will allow applications to trade data. Informatica makes extract, transform and load, or ETL, tools. ETL software copies data from one database into a database or data warehouse, where it can be analyzed.Traditionally, there has been an artificial barrier between the two, said Jim Ivers, senior director of product marketing for WebMethods. Information flows through enterprise application integration tools in real time, whereas ETL tools work more slowly because data is uploaded to the warehouses in batches, usually during periods of downtime. "The Informatica [solution] can capture any information flowing through our integration network," Ivers said. "We have taken the ETL process with its built-in latency and drawn it into the real-time world."SeeBeyond Technologies Corp., Monrovia, Calif., another middleware company, is also adding more business intelligence into its middleware products. In July, the company plans to introduce its own ETL tool for its middleware products. "If you look at enterprise customers who have purchased SeeBeyond, they all have an ETL component," said Sam Maccherola, vice president of public sector for SeeBeyond. He said the company offered business intelligence from partners, "but it made sense for us to bring it in-house and incorporate it into our platform."Requests for proposals often will ask for a specific analytics requirement, along with a need to bridge to legacy systems. Maccherola cited the INS Entry-Exit System that DHS is ramping up. "Entry-Exit has requirements for integration and ETL Web services," he said. "As they try to integrate all these back-end systems across all these different components, all of these capabilities are needed."Why not go with a best-of-breed solution? An integrator's strength, after all, is its ability to pick the strongest solutions for each part of the puzzle. Vendors said the advantage would be in having one company offering multiple parts."If I'm an integrator, and I'm trying to put together a solution for Entry-Exit, I have to cobble together all these various components into one seamless solution for my customer, and that can be very difficult," Maccherola said. "SeeBeyond provides one platform where everything is built on top of the same stack of code." Informatica and WebMethods are also pitching their combined offering as a headache-reducer. Sanjay Poonen, vice president of worldwide marketing at Informatica, estimated the Informatica-WebMethods solution, which typically costs about $500,000, will be able to save about $1.5 million to $2 million in labor.Although the integrator may lose out in charging for installation services, it can use the savings to bid more competitively on contracts, Poonen said. "We capture a little more in licensing fees, and a little of the services fees are lost, but a systems integrator can tell an agency chief information officer that it has a solution that can work faster, cheaper and with less risk," Poonen said.IT consulting firm Gartner Inc., Stamford, Conn., has coined a new term for this emerging market: business activity monitoring. It is a term that at least one new company is using to define itself. Celequest Corp., Redwood Shores, Calif., launches its new business activity monitoring software June 30. Founded by Diaz Nesamoney, a co-founder of Informatica, Celequest has thus far has attracted $6.5 million in startup funding.Real-time analytics could allow the Centers for Disease Control and Prevention to monitor for outbreaks of diseases, Nesamoney said. A business activity monitoring solution also could be used by the military to monitor the location of troops for treaties that require a certain number of troops be present in a given area. Nesamoney is betting that companies dedicated only to business activity monitoring will outpace middleware vendors that are branching into this arena. His company's product can work with middleware offered by WebMethods, Tibco Software Inc., Palo Alto, Calif., and others. Running on IBM Corp.'s WebSphere or BEA Systems Inc.'s WebLogic servers, Celequest software doesn't use a data warehouse or even a hard-drive-based database. It aggregates all the information on random access memory-based cache. "We don't actually store the data in a database. We process it in a stream, so there is no latency," Nesamoney said. *Staff Writer Joab Jackson can be reached at jjackson@postnewsweektech.com.XXXSPLITXXX-As agencies think about how to build systems to analyze large amounts of database information, they are usually split between using one of two architectures, said Allen Shay, president and chief operating officer of Teradata, a division of NCR Government Systems Corp., Dayton, Ohio.On the one hand, an agency or an integrator could leave data distributed across multiple systems, building a decision support system that draws data directly from these source systems. On the other hand, the agency could build a data warehouse, which compiles all the data from each of the systems into a separate database. Shay, whose company sells data warehouse solutions, extols the virtues of that approach. The reason is simple, he said. Transactional systems were set up for a specific purpose, such as processing payroll or parking tickets. Serving up additional data for a decision support system unnecessarily burdens them with a task they were not designed to do. It also presents the integrator with the daunting task of setting up multiple connections among unlike systems. "What happens in developing an enterprise warehouse is that we identify the formats of that data. We make sure that it comes into a common repository with a format that can be queried," Shay said. "So when you ask a question, you are asking it against not only a variety of data sources, but against data that is clean." Teradata's government customers include the state of Texas, which uses a data warehouse to ferret out delinquent taxpayers, and logistics offices of the Air Force and the Navy. Other vendors pitch the alternative. For instance, Crystal Decisions Inc., Palo Alto, Calif., offers decision support software that can analyze data from multiple sources without using a full-fledged data warehouse. "We don't have the robustness of a full data warehouse," said Matt Dion, senior products manager for Crystal Decisions. Instead, the company boasts that its report software can get up and running very quickly. It can draw data from transactional systems, such as PeopleSoft, during the slow hours, allowing users to generate reports and queries soon thereafter. It also can draw from on-the-fly data that is generated by systems that aren't entered into databases at all. "What we see is a lot of customers using our stuff on a tactical basis while they are on a more strategic route to a data warehouse. Sometimes they get there and sometimes they don't, but our stuff can be employed right away," Dion said.Earlier this year, Crystal Decisions struck a deal with the Naval Undersea Warfare Center Division to unify multiple data sources into one knowledge management system. Integrators and analysis tools vendors tend to be neutral on which approach works best."We support both equally as well," said John Cronin, vice president of the government sector for Autonomy Corp. plc, Cambridge, U.K. Autonomy's software categorizes, summarizes and personalizes organization information. Data warehousing is useful when the information is scattered among different agencies. "Our whole approach is that data can reside in many different places. Seldom does one organization have complete control over all the information it uses," Cronin said. "We've worked both sides," said Ken Bartee, president of integrator McDonald Bradley Inc., Herndon, Va, which uses data warehouses through its visualization work. "We've implemented several solutions where a middleware product can draw from eight or 10 databases.""We don't believe that reformatting the data in a data warehouse is always an answer. In some instances it is, but a lot of clients can work with a cheaper solution," Bartee said.