Not Your Father's Old Flat File

P> LOS ANGELES -- Data warehousing is a lot like those old flat-file databases that got everyone agog in the '80s -- except bigger, much bigger, and relational, very relational. If databases of the late 1980s were Lake Tahoe, the data warehouses of the late 1990s are the Pacific Ocean. Already, data warehousing is pooling the medical details of generations of Americans, from hang nails to heart attacks, and relating even those two.


But data warehousing is not merely a story of how databases got really big and complex.

These new gigabyte- and even terabyte-capacity data warehouses are also faster, much faster. The search engines on Tahoe were pretty basic outboards that would get a user across and back. Back then, finding the right data could be a lot like fishing -- takes awhile, even if you know where to drop your line. Now, for governments and companies with the money to buy into data warehousing, the churning digital ocean of data, ever-fed by fresh rivers of new data, is something to be grasped for practical use, not awed at in gee-whiz bull sessions.

The transition is huge. Take fishing. In data warehousing, the database operator sits in a stealthy jet zipping low over the chop. Inside the cabin, the operator can find the fish by intricate high-tech radar that half has a mind of its own for digitizing the location and projected movements of fish. In seconds, he knows where the big schools are going. Seconds later, he has a report that means something and is hard to argue with. Mission accomplished, the analyst zips back to LAX airport. As soon as the jet comes to a standstill, he can fire up his cellular fax/modem and broadcast a mass fax detailing trend analysis of fish school patterns up and down the West Coast to anglers willing to pay for the edge that data warehousing brings to analyzing everything from fish patterns to chances of being snagged yourself in an insurance scam.

The short story is that fishing for meaningful data -- once a half instinct, half high-tech endeavor -- has gone cold high-tech. In the brave new world of data warehousing, patterns are ascertained from on high -- or at least, from the glow of a computer terminal. Where the fuddy old outboard guys from the flat-file database times saw fish, the low-flying, stealthy geeks of the late 1990s see schools of fish and can trend the patterns where they're headed.

That analogy aside, the essential point is that those who know how to work an accurate data warehouse now have both the goods -- the data -- and the fast search engines to turn an ocean of intelligence goop into hard currency and high power like never before.

The bottom line: Data warehousing -- expected to rise to a $2 billion market this year -- is if anything a tenacious and never-going-away power resource for those who analyze and manipulate data. Those companies in a position to marshal data will make a killing "warehousing" it for high-paying subscribers.

This raises a larger question (see main story): Does the fact of massive data warehousing shift practical control of how to use intelligence from the craftsmen in the field -- doctors, lawyers, accountants, for example -- to some insurance or in-house factotum with an ocean of data at his fingertips and a 15-inch monitor in his face?


Reader Comments

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above

What is your e-mail address?

My e-mail address is:

Do you have a password?

Forgot your password? Click here
close

Trending

  • Dive into our Contract Award database

    In an exclusive for WT Insider members, we are collecting all of the contract awards we cover into a database that you can sort by contractor, agency, value and other parameters. You can also download it into a spreadsheet. Our databases track awards back to 2013. Read More

  • Navigating the trends and issues of 2016 Nick Wakeman

    In our latest WT Insider Report, we pull together our best advice, insights and reporting on the trends and issues that will shape the market in 2016 and beyond. Read More

contracts DB

Washington Technology Daily

Sign up for our newsletter.

I agree to this site's Privacy Policy.