Data Warehouse Market Draws a Crowd

Like an episode on the popular TV series the X Files, you can just sense it coming, the "universe = library" equation, the be-all, end-all information convergence, the ultimate data warehouse that engulfs all the bits and bytes that cross the routers.

You can visualize it by linking the Internet and its vast resources, the intranets and their transactions, historical data and mounds of digitized information, the hardware and software to store, manage, manipulate and mine the data and the clients, such as Web browsers, to easily retrieve and present the results for knowledge workers.

But that's a vision of the future. Where are we today?

On its Web page, Informix, one of the key data warehousing players, describes a data warehouse now as a flexible environment made up of technologies that take your organization's operational, historical and external data, consolidate it into a separately designed relational database, manage it and mold it into a subject-oriented format that is optimized for end users to access and analyze.

The intended result? To offer better customer service, create greater customer loyalty and activity, focus customer acquisition and retention on your most profitable customers, achieve customer intimacy, increase revenue and reduce operating costs. In other words, become more profitable and competitive.

For Conrad Moses, Oracle's national director for data warehousing in its Government Services unit, a key aspect of data warehousing often overlooked is its importance in business process re-engineering and its effect on the end user.

"Government agencies are taking seriously the president's directive to make information available to the public. A good example is the U. S. Environmental Protection Agency's EnviroFacts effort. That effort involves re-engineering as federal executives develop strategy, managers coordinate change and staff redirect resources from mainframes to networked distributed computing and review the quality of the information -- all for the benefit of the end user," says Moses.

The technical goal of data warehousing, according to The Data Warehousing Institute, Bethesda, Md., is to resolve data access difficulties that were recognized for many years, but remained insurmountable because suitable technology was lacking. Among the key problems that the trade organization identified were:

-Unavailable data hidden in transaction systems

-Interminable delays as underpowered systems tried to perform huge, complex queries

-Complex, user-hostile interfaces on software

-Difficulties in discovering patterns in large amounts of data

-Administrative costs and complexity in supporting remote users

-The competition for computer resources between transaction systems and decision support systems

Now, notes the institute, we have the storage, along with fast, inexpensive parallel processors and new analytical software, enabling us to fashion thousands of data warehouses. More user-friendly query and analysis tools let more professionals exploit data warehousing. And the data warehouse is integrated with popular tools such as the World Wide Web and data mining applications.

Whatever your view, it's a big market attracting the computer industry's key players. The Gartner Group, International Data Corp., Meta Group, IBM and others forecast a sizable marketplace ranging up to $30 billion by the year 2000. The Meta Group, a California consulting firm, estimates that the 400,000 users of what IBM terms business intelligence information today in the United States will explode to 10 million "knowledge workers" by the turn of the century.

Why all this excitement? Because many see the end user finally victorious over the data and organizations exploiting a previously unmined asset as we enter the knowledge age. Is data warehousing all it's made out to be? Or is it the infotech version of alchemy, where the data warehouse is the universal solvent and the elixir of life? Only time and money will tell. In the meantime, there are big spenders at the table bidding up the stakes in an all-night poker game.

MicroStrategy

Vienna, Va.

Among the current high flyers is MicroStrategy, which some consider the leading provider of relational online analytical processing. ROLAP is a tool for performing multidimensional analysis on large data sets. The $25 million company, which reputedly controls 60 percent of the ROLAP market, recently launched its 4.0 product line, marketed as the most comprehensive OLAP solution on the market. It includes a dynamic Web offering, an Excel add-in, an OLAP report writer and VLDB drivers. MicroStrategy's vision of the future is no less expansive -- to bring the data warehouse to the consumer.

Manish Acharya, MicroStrategy's director of marketing, sees the evolution of data warehousing from departmental solutions to the enterprise, and now with the Web, beyond the enterprise. The short-term result? Cross-enterprise and commercial business applications. Yet, he sees more potential in the long term, when companies open their data to consumers worldwide.

Imagine, for example, a politician's professional staff studying the demographics of a data syndicator's data warehouse to decide where best to target a message. Or a patient in need of surgery connecting to a medical insurance data warehouse to find the best doctor in the area.

"The consumer data warehouse could potentially lift this industry from a few hundred million dollars to hundreds of billions of dollars," says Acharya.

The company gained broad exposure by working with Giant Food Corp. in helping re-engineer its store operations. MicroStrategy's DSS Agent, a ROLAP tool, was used to select, analyze and present historical data for category management, a method of researching, buying and selling groceries.

Cognos Inc.

Burlington, Mass.

A company focused on business intelligence tools to access, extract, analyze, use and present data warehouse data is Cognos, with its PowerPlay and Impromptu. Not surprisingly, its products work with Oracle and Sybase databases, as well as with data marts -- small-scale, subject-specific data warehouses. Cognos likes to compare data warehouses to plumbing and their tools to the plumbing fixtures, the faucets and relief valves.

Holistic Systems Inc.

Edison, N.J.

Another approach to business intelligence comes from Holistic Systems with its Holos 5.0 product, an application development environment to access, analyze, perform multidimensional modeling and report on data. Holos features a broad range of options for storing and analyzing multidimensional data, from ROLAP to MOLAP (multidimensional OLAP). It also contains fully integrated mining capabilities, intelligent agents, an advanced application development manager and the ability to configure both client and server on a single NT PC. It is available on most UNIX and VAX/VMS servers, with native windows, Windows/NT and Macintosh desktop clients.

Microsoft Corp.

Redmond, Wash.

Of course, one can never overlook what Microsoft is doing in the computing arena. Its data warehouse direction focuses on using the Microsoft SQL Server. Given its platform-specific approach, the company's strategy pinpoints alliances with leading industry vendors and independent software vendors for data warehouse solutions.

According to corporate materials, the Microsoft strategy includes:

-Active Data Warehousing Framework, an extensible set of interfaces to simplify design, integration and management of data warehouse solutions

-Microsoft Alliance for DW, a coalition to gather the industry's leaders in data warehousing and applications

-Product enhancements to Microsoft SQL Server, including new features for enhanced query processing, information delivery, data transformation, Active Data Warehousing Framework interfaces and Internet integration

-A service offering from Microsoft Consulting Services to enable customers to plan, design and implement data warehousing solutions based on Microsoft SQL Server

NCR Corp.

Dayton, Ohio

One company that jumped on the Microsoft DW Alliance bandwagon early is NCR. It just announced its Teradata Relational Database Management System to bring the benefits of the Internet and multimedia forms of information to the warehouse. It will port Teradata to Microsoft's Windows NT and add multimedia and Internet enhancements to Teradata. The Teradata port to Windows NT also positions NCR to sell Teradata on its WorldMark servers running NT, as well as other hardware platforms. That event for SMP (symmetric multiprocessing) systems is slated for the fourth quarter of 1997 and for MPP (massively parallel processing) systems for early 1998.

The company also announced that Teradata will support dependent and independent data marts. A dependent data mart is a data warehouse organized by subject area or user group and sources its data from the enterprise data warehouse. Independent data marts are also organized by subject area or user group, but derive data from operational systems. NCR's data mart initiative includes enhanced, high-speed data replication services for Teradata that distributes data in both asynchronous and synchronous modes. Playing all its cards, NCR earlier this year announced a partnership with MicroStrategy to offer customers remote access to their data warehouses through the Web.

Pilot Software Inc.

Cambridge, Mass.

In the data mining area, looming as the next major segment, Pilot Software Inc., a company of the Dun &amp Bradstreet Corp., has its Discovery Server, the mining component of its decision support suite targeted to sales and marketing executives. It links with any data warehouse for predictive modeling and high-level visualization in solving complex marketing problems. Using algorithms for predictive models, the tool yields highly visual market segmentations that can be used to target customers. One application is to test and implement marketing ideas. Moving toward the end user, the server highlights the most important customers and customer information in a data warehouse and presents results in plain English.

It's all about managing customer relationships, or as Alan Paller of The Data Warehousing Institute says, "achieving customer intimacy." Ultimately, it means helping sales and marketing professionals leverage key customer information, supporting the company staff in getting and keeping clients.

slp InfoWare

Paris, France

The international data warehouse arena is also active. As one example, slp InfoWare recently opened two offices in the United States to sell its full suite of data warehouse products and tools, as well as data mart applications, offering an integrated, cross-platform, client/server solution that includes data mining, query and reporting, and multidimensional databases. Among its better-known customers are France Telecom, the world's fourth largest operator, Hewlett-Packard Co. and McDonald's Corp. Currently, the company is focused on U.S. vertical markets in health care, telecommunications and banking.

Alongside the data warehouse functions is the issue of storage. Today, EMC Corp., Hopkinton, Mass., with its strategic partner, NCR, holds the record for the world's largest data warehouse, an 11-terabyte solution in Tokyo. The 11 terabytes equals about 2.75 billion pages of text. And the company has teamed up with the likes of Oracle, Informix and Sybase on joint technology projects.

The company's flagship product, Symmetrix 3500, can store more than a terabyte in less than 17 square feet and is based on EMC's Integrated Cached Disk Array design. The company claims to be the leading supplier of intelligent disk arrays for the IBM-compatible mainframe storage market and the leading independent supplier in both the open systems and IBM AS/400 storage markets. It also offers the Centriplex system for client/server platforms.