Demand for real-time data spurs middleware

The federal government's push for better information sharing among agencies, made more urgent by e-government initiatives and the war on terrorism, has created strong demand for enterprise software architectures, Web standards and integration tools. Among the latter, that old chestnut middleware has been dusted off. Agency information technology managers are searching for better ways to integrate not just the applications and legacy databases of far-flung offices, but also the up-to-the-minute wireless communications of border agents, real-time inventory levels in defense procurement and logistics systems and satellite weather feeds that mean go or no-go to waiting bomber pilots.Middleware projects are rampant at the federal level. The Army Aviation and Missile Command, for example, used webMethods Inc. middleware to pull together legacy technical manuals and other critical information on Blackhawk helicopters to display on a secure Web site.Other Defense Department divisions are working with middleware vendors to build visibility and secure messaging into the department's emerging worldwide supply-chain network.Definitions of middleware have evolved, but at bottom they've always been what their name implies: software that sits between two systems, usually applications but sometimes networks, so the systems can communicate directly. Historically, there have been two main kinds. The first, messaging-oriented middleware, also known as enterprise messaging, swaps generally simple messages between systems. The second, adapters, employ richer templates that map data between the two. As the Internet became the network backbone, Web standards such as Extensible Markup Language, Java and its messaging protocol, Java Message Service, and related Web services have been added to bring more universal data sharing among an exponentially growing number of sources.Middleware products have roots in four infrastructure technologies. Messaging-oriented middleware platforms and transaction processing monitors such as IBM Corp.'s WebSphereMQ, Tibco and BEA's Tuxedo started decades ago as mainframe and Unix integration tools. Another vein is enterprise application integration, a hot category since the late 1990s that has consolidated down to a handful of companies, notably webMethods, Fairfax, Va., Vitria Technology Inc., Sunnyvale, Calif., SeeBeyond Technology Corp., Monrovia, Calif., and Tibco Software Inc., Palo Alto, Calif. Enterprise application integration companies made their names building adapters between popular applications, such as Oracle Corp. databases and SAP America Inc.'s R/3 enterprise resource planning, and packaging them in a single product. They often include messaging-oriented middleware in their integration suites alongside the adapters. More recently, makers of Web application server infrastructures, most prominently BEA Systems Inc., San Jose, Calif., and IBM, Armonk, N.Y., have sold integration suites geared to their application servers. They've been successfully pushing three-tiered infrastructures that match application servers with integration servers, both powering a third, portal-server box that presents the information collected by the middleware in a single browser page. Enterprise application integration vendors argue, with some justification, that their integration works with a broader range of platforms.The newest category contains companies such as SeeBeyond, Software AG of Darmstadt, Germany, and Sonic Software Corp. of Bedford, Mass., that have planted their integration flags in the open Web standards of XML and Java. "It's a little bit different from classic EAI," said Joe Gentry, Software AG's director of product marketing. "It really is designed around a standards interface."The accompanying chart includes the main offerings from most of these middleware vendors. It does not show vendors, such as BMC Software Inc. of Houston, Candle Corp. of El Segundo, Calif., and MQ Software Inc. of Minneapolis, that sell management tools and add-ons that augment core middleware.Besides Web standards, the hottest trend affecting middleware design is an effort to relate low-level integration tools to the business processes and people who use the applications. The idea is to employ graphical user interfaces, diagrams and other executive friendly tools to redesign workflows; some middleware, including webMethods', comes with workflow tools. "First there was EAI, where you connect application A to application B," said John Kiger, BEA's director of product marketing for the WebLogic enterprise platform. "Connecting A to B was no longer sufficient. You also needed some business logic that would represent the business process that was the driver for integrating A to B." More recently, business activity monitoring, so-called dashboards for real-time monitoring of the business processes running on middleware, has begun to be incorporated. "BAM is a real-time representation of the operational metrics of your business," said Jim Ivers, senior director of product marketing at webMethods. Integration among vertical layers is starting to take other forms. For example, webMethods Manager can tell you which business processes are affected by a server crash, according to Ivers.Along with the demand for integration far beyond internal local networks, the push for open standards has led to a new vision of middleware as a more flexible backbone infrastructure for all sorts of application and network integration. Vendors and analysts have new names for the phenomenon: enterprise service bus, or ESB, and service-oriented architecture, or SOA. ESB implies an XML-based bus analogous to hardware buses on PC motherboards, and into which you plug adapters for specific applications. Ivers likened the adapters to vertebrae in the backbone. "ESB is a very important topic," said John Rymer, research vice president at Giga Information Group, a market research company in Cambridge, Mass. "It addresses one of the problems with middleware: They tend to be very isolated." SOA, in contrast, is a catch all for middleware that works like a Web service, with applications sharing messages while hiding their internal workings from each other. Middleware vendors have jumped on the Web services bandwagon by supporting the key industry standards promoted by the World Wide Web Consortium. But even the staunchest Web services proponents, including IBM and Sonic Software, admit the standard is weak in two areas ? transactional integrity and security ? that drive people to traditional middleware in the first place. While formerly an e-commerce issue, secure transactions are increasingly important for emerging homeland security applications. Immigration officers or local police inputting fresh criminal records from the field, for example, require guarantees that remote messages are delivered and properly stored in central databases. However, the World Wide Web Consortium is working on specifications to address the concerns. Regardless, Web services with comparable transactions to traditional middleware are probably 12 to 18 months away.A couple of years ago, "there was sort of a general feeling that Web services would do away with integration, but in fact it's been a driver for integration," Ivers said, citing confirmation from analysts such as Gartner Inc.