Cyril on Business Intelligence

There is a lot of consultant self-righteousness out there on the issue of data quality and its impact on the ability of enterprises to build SOA environments generally and effective Business Intelligence reporting specifically. The newsletters are full of the stuff. Mostly its crap, and we all know it. Checklists upon checklists of common sense. Death by newsletter regurgitated bumpf to fill out the ads!

In my opinion there is too much beating of breasts and doom and gloom talk – we’ve always had bad data, and, let’s face it, always will. I don’t condone bad practice, but we do need to keep the perspective – perfect data will cost an infinite amount of resource, and much of it will never be looked at, let alone used in a way that causes grief.

Remember Method One? If you’re old enough in the business, you’ll recall how we had milliions of forms to fill in, checklists to check, signoffs to get. Basically, you either did your job, built the system, or, alternatively filled in the forms and did nothing useful.

Surely the best course for the smart Business Analyst is to know:

What you have in the data warehouse

Where it came from

Who uses it, where

If you think I’ve got it wrong, let me know. But, the path I’m taking in our BI work is to ensure that the metadata is documented, in diagrams and tabular form, and the lineage to and from the cubes and tables of KPIs and measures is easily traced. The same should apply to Service Oriented Architecture projects, everything is desirable, but we should only do what is essential, there’s enough of that.

If a report is queried, there’s a dispute over accuracy or relevance, then check the sources and trace the history. It will soon be clear if there’s a problem or not. So do the sensible thing, cover the obvious issues, and rely on the documentation for the issues that arise. Of course, the documentation has to be right – but this is reasonably easy to do.

Good practice combined with this facility will surely be the cost effective option. I hope to be more prescriptive in the near future. Watch this space, or check the progress on how I’m “operationalizing” this at www.bidocumenter.com.