Drowning in Data

Dec. 1, 2003
Drowning in Data Decisions, reports, data collection . . . ever feel like youre drowning in data? Youre right. According to a recent study from the University

Decisions, reports, data collection . . . ever feel like you’re drowning in data? You’re right. According to a recent study from the University of California, Berkeley, worldwide information production has increased by 30 percent each year between 1999 and 2002. Research done by BuzzBack Online Market Research for Teradata showed similar findings. “It’s not knowledge, just data,” said Professor Peter Lyman of the University of California.

Our society is collecting unprecedented amounts of data on just about everything. Financial legislation, disaster recovery implementations, real-time business communications and the Internet economy are contributing huge volumes of data to businesses, claimed Jeff Jenkins of Hewlett-Packard in the report on the University of California study. On top of these factors, you have businesses’ new requirements to integrate and share data across the enterprise and the supply chain.

The programs that have been available for the last decade or so, that “drill down” into data, offer up data in pretty raw form — often just numbers on a screen. It’s been up to executives to analyze all of it.

Thus, it’s no wonder executives feel as though they are drowning in data. Additional findings from the BuzzBack survey show that executives feel that key decisions are more complex while the time to make these decisions has decreased. Plus, more than 75 percent of the 158 executives questioned feel that poor decision-making will affect profits and revenue, the corporate reputation and morale and profit margins.

Is there a solution to this problem? There are sales pitches about possible solutions. Keep in mind, though, as Professor Lyman noted, that it will take thoughtful people using smart technologies to figure out how to turn data into information and then how to make sense of the findings. This is no small task.

Executives are looking for information to be presented in what’s being called an “executive dashboard” format, a quick, graphic glimpse at how well each division in a company is doing at any given moment. Included in that dashboard will be recent historical data on each division, so executives know moment to moment whether a division is profitable or not. Material handling and the supply chain will be key data collection spots because the goal is for a corporation to “close the books” daily.

The latest programs that go beyond drilling down can “mine” and “warehouse” data. These programs combine artificial intelligence techniques with statistical modeling capabilities to tell executives the probability that Sue Customer will purchase five pair of shoes this year, spending about $400 total.

These programs, known collectively as Business Intelligence (BI) systems, have been working in scientific research and academia since the 1990s, and are now entering the business world.

Like anything that sounds good, there’s a downside. There are issues to grapple with, such as consumer privacy concerns discussed in November’s column.

And here’s another critical issue: integrating and sharing data across the enterprise, including the supply chain. Sounds a bit familiar, doesn’t it? Material handlers have been dealing with many of the integration and sharing issues for years. And many of you are still working on the problems.

Will these BI programs be the answer for all parts of an enterprise? Analysts point out that executives will still have to deal with cleaning data, eliminating duplication of data, and merging data silos. Let’s not forget the problems of poor communication of vision and resistance among territorial managers. And, from a technological perspective, there will be predictable problems (you don’t need a BI system to tell you this) with purchasing low cost systems and disparate and unrelated technologies.

BI systems could be the life preserver executives are looking for, but, despite the glowing sales pitches, it won’t be a simple matter of throwing one to the drowning executive. Implementation of BI probably won’t be point-and-click simple. The programs themselves are easy to install; however, the corporate infrastructure for their operation may not be in place.

Thus, executives still need to be good swimmers.

Leslie Langnau, contributing editor

[email protected]