Cyril on Business Intelligence

The revolution caused by VisiCalc in the 70s is still running its course, as I outline later. These days in the BI context there seems to be one of us doing something for every 10 telling us what to do, how to do it, how not to do it, why plan for it, what to buy to do it, etc. This reminds me of Bertrand Russell’s famous observation:

“Work is of two kinds: first, altering the position of matter at or near the earth’s surface relative to other matter; second, telling other people to do so. The second kind is capable of indefinite extension: there are not only those who give orders, but those who give advice as to what orders should be given.”

The number performing the second kind is far greater than those “at the coal-face”.

Last post, I considered the impact on BI of five proposed discontinuities nominated by Gartner that were bearing on IT generally. As is appropriate for the second class of workers lecturing the first, we were then told what to do about them, again with a set of five - the optimal number of slideshow points, it seems.

How should we take this advice? Well, first of all what were we advised to do?

Question Core Assumptions about the Role of the IT Organization

Experiment with Free-form Environments

Help Users Innovate

Segment Users (according to need and importance)

Stop Trying to Provide Everything (i.e. all users aren’t equal)

To my “first kind of worker” mind, these imperatives are really plays on the same theme: BI system users want to be empowered, or will empower themselves, but have varying degrees of skill, ability and needs.

I believe that demand will cause Rogue Users to overwhelm the conventional BI frameworks. I suspect that this has huge implications for the Big Three and their takeovers of Hyperion, Business Objects and Cognos, but maybe they don’t yet know it. User rebellion at the humongous complexity that must result from these takeovers is almost a certainty. The major vendors have the classical bind of needing to protect the base while catering for the new order; in the same way as land line based telephone companies need to protect the base while competing against low cost internet and cell phone carriers.

The answer, of course, lies in the data management.

Quoting from a blog entry by Cindi Howson (based on an Italian experience)

“Thousands of stand-alone spreadsheets disconnected from the BI environment continue to wreak havoc on everyone. Users spend an inordinate amount of time debating whose numbers are right rather than focusing on the business problems at hand. As one person expressed their frustration, “One of the most damning things BI vendors did was to allow direct export to Excel.” In defense of the vendors, users asked for this! In defense of the users, it’s because BI is not always deployed in a way that meets users needs. But just as parents have to say “no” to children, sometimes vendors and IT should say “no”, it’s not in the best interests of the company.”

I disagree that the vendors are driving the situation. Customers are always “right” when this situation exists.

I feel for the vendors, sort of, but this is simply a repeat statement of the VisiCalc Syndrome. If you were professionally active in the 70s, Dear Reader, you will recall the chaos that started this spreadsheet style of BI, Apple II and VisiCalc spreadsheets. Suddenly, we High Priests of IT (or perhaps more accurately EDP) were no longer the critical path for data analysis.

Whatever you think of the merits of massive BI software, the explosion of spreadsheet use since the 70s has far outstripped the growth of BI. It is the foundation of future BI, however. Eyeballs count in new technology, and most of the eyeballs are watching Excel.

This is what makes the Microsoft SharePoint family of application support software so interesting. Will it become the focus for our Rogue Users to tame the spreadsheet anarchy referred to by Cindi Howson? I believe it will, and we will see the progressive, but gradual, atrophy of the large, integrated, BI application as per Cognos, Hyperion, and Business Objects.

Bertrand Russell has another well known quotation: “The only thing that will redeem mankind is cooperation.”

To paraphrase for our discussion: The only thing that will save BI is cooperative collaboration.

Of course, I am not advocating anarchy, I simply believe it is inevitable if we try and maintain the current direction. The path to sanity is surely user empowerment at the application and reporting end, and quasi-Nazi control at the data end. The reason the VisiCalc syndrome was so painful was that the analysts all had bad data, mostly keyed manually from EDP reports (often themselves based on bad data!).

Metadata management is the key task for the workers (the real ones that is) as I have discussed previously. Let’s do it.