The VisiCalc Syndrome Lives. And Does BI’s Future Depend On How We Manage It? November 20, 2007
Posted by Cyril Brookes in General, Metadata management.2 comments
The revolution caused by VisiCalc in the 70s is still running its course, as I outline later. These days in the BI context there seems to be one of us doing something for every 10 telling us what to do, how to do it, how not to do it, why plan for it, what to buy to do it, etc. This reminds me of Bertrand Russell’s famous observation:
“Work is of two kinds: first, altering the position of matter at or near the earth’s surface relative to other matter; second, telling other people to do so. The second kind is capable of indefinite extension: there are not only those who give orders, but those who give advice as to what orders should be given.”
The number performing the second kind is far greater than those “at the coal-face”.
Last post, I considered the impact on BI of five proposed discontinuities nominated by Gartner that were bearing on IT generally. As is appropriate for the second class of workers lecturing the first, we were then told what to do about them, again with a set of five – the optimal number of slideshow points, it seems.
How should we take this advice? Well, first of all what were we advised to do?
Question Core Assumptions about the Role of the IT Organization
Experiment with Free-form Environments
Help Users Innovate
Segment Users (according to need and importance)
Stop Trying to Provide Everything (i.e. all users aren’t equal)
To my “first kind of worker” mind, these imperatives are really plays on the same theme: BI system users want to be empowered, or will empower themselves, but have varying degrees of skill, ability and needs.
I believe that demand will cause Rogue Users to overwhelm the conventional BI frameworks. I suspect that this has huge implications for the Big Three and their takeovers of Hyperion, Business Objects and Cognos, but maybe they don’t yet know it. User rebellion at the humongous complexity that must result from these takeovers is almost a certainty. The major vendors have the classical bind of needing to protect the base while catering for the new order; in the same way as land line based telephone companies need to protect the base while competing against low cost internet and cell phone carriers.
The answer, of course, lies in the data management.
Quoting from a blog entry by Cindi Howson (based on an Italian experience)
“Thousands of stand-alone spreadsheets disconnected from the BI environment continue to wreak havoc on everyone. Users spend an inordinate amount of time debating whose numbers are right rather than focusing on the business problems at hand. As one person expressed their frustration, “One of the most damning things BI vendors did was to allow direct export to Excel.” In defense of the vendors, users asked for this! In defense of the users, it’s because BI is not always deployed in a way that meets users needs. But just as parents have to say “no” to children, sometimes vendors and IT should say “no”, it’s not in the best interests of the company.”
I disagree that the vendors are driving the situation. Customers are always “right” when this situation exists.
I feel for the vendors, sort of, but this is simply a repeat statement of the VisiCalc Syndrome. If you were professionally active in the 70s, Dear Reader, you will recall the chaos that started this spreadsheet style of BI, Apple II and VisiCalc spreadsheets. Suddenly, we High Priests of IT (or perhaps more accurately EDP) were no longer the critical path for data analysis.
Whatever you think of the merits of massive BI software, the explosion of spreadsheet use since the 70s has far outstripped the growth of BI. It is the foundation of future BI, however. Eyeballs count in new technology, and most of the eyeballs are watching Excel.
This is what makes the Microsoft SharePoint family of application support software so interesting. Will it become the focus for our Rogue Users to tame the spreadsheet anarchy referred to by Cindi Howson? I believe it will, and we will see the progressive, but gradual, atrophy of the large, integrated, BI application as per Cognos, Hyperion, and Business Objects.
Bertrand Russell has another well known quotation: “The only thing that will redeem mankind is cooperation.”
To paraphrase for our discussion: The only thing that will save BI is cooperative collaboration.
Of course, I am not advocating anarchy, I simply believe it is inevitable if we try and maintain the current direction. The path to sanity is surely user empowerment at the application and reporting end, and quasi-Nazi control at the data end. The reason the VisiCalc syndrome was so painful was that the analysts all had bad data, mostly keyed manually from EDP reports (often themselves based on bad data!).
Metadata management is the key task for the workers (the real ones that is) as I have discussed previously. Let’s do it.
Implications for BI from Gartner’s Five Discontinuities for IT November 4, 2007
Posted by Cyril Brookes in BI metadata documentation, Issues in building BI reporting systems, Tacit (soft) information for BI, Taxonomies, Tags, Corporate Vocabularies.8 comments
Those following the technical press will know of the five discontinuities to orderly IT business evolution as expounded by Gartner in Orlando in early October. I have no intention of adding my thoughts to the copious analyses and assessments of these as regards IT generally. However, it may be of interest to you, Dear Reader, to embark with me in exploring the Business Intelligence impact of these Famous Five.
The press release and examples of the commentary are here and here, but to make it easier to judge your interest level, the five movers and shakers envisaged by the gurus are: Web 2.0, Open Source Software, increasing consumerization, Global Computing and SaaS.
All of these factors have been considered in earlier posts for this blog, but in this one I will give a short overview of my opinions on the BI implications, and point to former, often more detailed, discussion.
The two immediately obvious observations I draw are:
SOA has finally been tipped out of the IT analyst headlines – and about time too. Thanks be!
The inter-dependence of the factors, and their cumulative impact, are at last being recognized.
I’ll comment on the Five in order of my perceived significance to the BI community.
Web 2.0 and Enterprise 2.0
Most of us already know that social networking has huge potential to reshape the BI environment. I’m not sure of the precise tools that will become dominant; I don’t believe they are in complete form as yet. However the key resource is, in my view, TACIT information. And the key process is SHARING. Sharing information, up till now almost solely done effectively in face-to-face or video/phone meetings, is the only way that knowledge is created.
Therefore, in my view, Enterprise 2.0 will become the new all purpose marketing term to replace Knowledge Management. This much is unavoidable, the vendors will see to that.
The importance of My Space, Facebook, Wikis, and their ilk is that they are rapidly reshaping the culture of communication. I have outlined my views on the cultural roadblocks to effective information sharing. The new generation of professionals, each one with his/her own personal net space, will presumably more readily accept collaboration imperatives from the executive suite. More readily, that is, than the current set of professionals who are decidedly reluctant to do so. I must admit that I do not think the culture will shift that much, and we still will have this problem in a decade.
It is in the interests of young people to share details of their interests, activities, etc. This is part of the current social norm. However, when corporate life intrudes older survival instincts may well prevail such as: the perceived power of knowledge (say, customer preferences, possible new clients, etc.); the desire to remain part of a team (not being the whistle blower); and the danger of being the messenger of bad tidings who is shot.
Up till now, it appears to me that the personal related Web 2.0 applications have little relevance in the corporate context, other than this culture shift opportunity. However, the evolution of Wikis is harder to assess.
Wikis are collaboration tools. They can be used as knowledge creation devices, provided that the classification, categorization and alerting mechanisms are effective. In corporate BI, as opposed to the social context, speed of response is critical. Knowledge must be created rapidly, and this implies almost instant initiation of collaboration processes that enable this creation.
Information that is not categorized cannot be searched or personalized for alerting.
I have often argued that a universally agreed taxonomy is important if people with information are to find people who can build on it, and those who need to use the knowledge created by collaboration. There is much talk of folksonomies, where classification is determined by the individuals. They are useful on the Internet, but I maintain that they are unsuitable for tacit information management. Newer categorization and retrieval software may well alter my opinion, but as yet, I believe that the taxonomy of approved, standard, preferred terms is an essential building block if wikis and similar knowledge stores are to be part of evolving BI applications based on Web and Enterprise 2.0 technology.
Open Source and SaaS
These are the tools of the Rogue BI professional I have blogged about earlier. For corporate BI these emerging movements, they’re not really technologies, have major implications. They’re not really discontinuities either, since they been evolving since timesharing terminals were invented some 40 years ago. And think spreadsheets, the most common Rogue tool at present.
Drawing from that earlier post:
I define rogue BI as the user self-creation of unofficial, unapproved corporate performance reporting, competitive analysis and customer behavior assessment.
To mix my metaphors: Blessed are the BI rogues, for they shall know what they want much sooner than IT will get around to filling their request.
The pressure for instant gratification in BI is intense, perhaps more so than the teenager’s need for the new Playstation game. Or maybe it’s just the grown-up version of the same.
Rogue BI is a management nightmare if left unaddressed by IT and senior executives. However, if managed properly, it will be a salvation, since much of the demand for new BI services will be redirected from conventional, and probably over-stretched, IT capability.
The solution for the over-stressed IT executive lies with effective metadata control and management. Provided the data is under control, the rogue can do little harm to the corporate entity. Of course, the rogue’s BI reports may be rubbish, but that ought only cause localized difficulty; as long as the data used is correct, of course. But then, any reporting from crook data is going to be rubbish.
The Rogue BI movement, driven by Open Source and SaaS, will probably have most impact on the vendor community. Heavy hitting BI software will have to become far more user developed application friendly. It will be a difficult shift for the marketers, as it threatens product differentiation. But it must happen. Spreadsheets are already there.
Consumerization
Dreadful word, it is, but it’s the one Gartner use in the press release, so who am I…. It relates to the common, now anyway, problem of system developers generally that uses are so used to high class presentation from their internet experiences that they demand the same quality user interfaces and capabilities from their in-house corporate BI systems.
If the Open Source and SaaS vendors (is an Open Source supplier also a vendor?) capitalize on this user demand then it will be another factor in promoting Rogue BI. And with good justification too.
Global Computing
I’m not terribly sure what the gurus mean by this. Perhaps they wanted an odd number of discontinuities? The most common example given in the commentaries is “Google Apps”.
I assume that the main implication to be drawn for BI proponents is that the pervasive and high response nature of Google Earth etc. is giving us new opportunities for BI projects, outside the normal scope of BI reporting applications.
I guess that says it all.
What else is there in the BI discontinuity department?
Well, the drive to automated systems doesn’t rate a Gartner mention, but undoubtedly the new rule management systems are discontinuities in the BI environment as I explored in some detail a few weeks back. Automation in IT generally is taking a back seat, but it’s full steam ahead in the BI space, see, for example, the work of James Taylor. There is a consequential imperative for IT and BI analysts. Revisit your current set of BI applications to assess where the automated decision making capabilities can be used profitably.
The rise of capability with unstructured information systems covered in my last posts (here and here) are also important. Web 2.0 is part of the story, but there’s much more technology at work in the move to enhance BI applications with both explicit and tacit unstructured information.
Next post, I’ll consider the BI related management implications of these trends, since Gartner’s symposium presentations have also generated considerable comment on how enterprises ought respond to the discontinuities.
