Cyril on Business Intelligence

These are the kinds of questions we ask our children – where were you, what happened last night?  And to the putative, hopeful, BI support analyst and developer, the DW content, the cubes, the dimensions; they are our children.  Strangely, the answers are just as hard to get on metadata as with the kids, or at least the right up-to-date answer.   

The issue is going to get much worse.  Why?  Because we are rapidly moving from BI “Greenfield” design situations to “Brownfield” support and re-design environments.  We’re stuck with  the treadmill of history as well as client executives who want better BI. 

There are two different sets of metadata related problems here:

• Support of operational BI systems, and
• Design of new or improved BI 

But, they both have the same solution. 

Everything is related to the BI metadata, sql database documentation and associated BI development system tables, cubes, etc.  This is a problem to us all because: 

• Metadata is not in one place – it’s a real dog’s breakfast; even when all the BI tools come from the same vendor
• It is dynamic, so we need to be able to monitor the sql documentation to see what the DW architects have done since last time we looked
• Comprehensive commercial repositories, at least those available today, are cumbersome and not kept current
• Synchronization of the metadata component documentation is super-important to the analyst – without it your project must surely fail.
• BI design is not concerned with much of the metadata available for a database, the focus is on the business related items and not the process type – schedules, update counts, etc.  Often, the unrelated stuff dilutes that which is important.
• Metadata standards are too complex, and not a useful categorization tool for BI analysts at present
• Essentially, the issues for BI analysts are:  What is in the current DW, the cubes, KPIs, tables, columns, dimensions, reports, etc.; and where did they come from.  Most database and BI software analysis tools are inadequate for this purpose. 

Support analysts find it difficult to track down the sources of bad data.  Clients complain about inaccurate reports, but why is it so?  We need, indeed must have, better BI related metadata repositories.  We don’t want all the crap, pardon my expletive, just the business related stuff, so we can find out …..Where it came from and where has it been – to make such lousy, and inscrutable, content. 

That’s probably enough stating the issue for support analysts. 

Next post I’ll detail how these same issues impact the BI developer, especially the person trying to improve on an existing BI system.

I’ll then outline my proposed solution, a specialist BI metadata repository tool.