jump to navigation

Unstructured Information in BI – Implementation Practicalities with Tacit Data August 30, 2007

Posted by Cyril Brookes in General, Tacit (soft) information for BI, Taxonomies, Tags, Corporate Vocabularies, Unstructured Information.
add a comment

Designing an unstructured information based BI system must take account of the explicit and tacit distinction. There is consensus for this, in blog-speak that means “me and my mate in the next office agree”! Feedback on my last two posts does, however, unanimously support this contention. The issue remains, however, so what do we do about it? Here’s what I propose.

Most businesses have a reasonably adequate process for collecting explicit unstructured information, the documents, news, emails, reports, etc. And if your’s doesn’t, the corporate portal experience awaits your attention. For heavy hitters the UIMA approach with its multi vendor retinue is available, and willing, for a substantial sum.

I have opined in the last posts here and here that explicit unstructured information is not where BI relevance is at. It can be a start, but the real value lies in the qualification that the executive and professional mind-space can give to seeds of BI, both explicit and tacit. The tacit realm is the goldmine; it is where the current, relevant, actionable, validated business intelligence lies.

How, then Dear Reader, do you capitalize on your tacit resources?

It’s a 9 step process, as I see it

  1. Encourage contributions from everyone, everywhere, based on credible rumor, opinion, assessment, etc.
  2. Scour the corporate world for knowledge building seeds, explicit and tacit – web crawlers, internal and external portals, news feeds, etc.
  3. Selectively disseminate raw data seeds to subject specialists – formally appointed for preference
  4. Encourage comments on those seeds by the specialists – acts, sources, cross-references, importance, time criticality – with discussion threads escalating in importance as appropriate
  5. Selectively disseminate comments – dynamic audience creation, so that more people, and more senior executives, are aware of more important issues
  6. Encourage issue identification by executives and professionals – implications, assessments, importance value adjustments, criticality adjustments
  7. Selectively disseminate the discussion – dynamic audience modification as business significance becomes clearer, possibly creating closed group discussions if the issue becomes strategic
  8. Propagate decisions made to the appropriate staff
  9. Store knowledge created – with time stamp, sunset clause if appropriate, to help avoid multiple solutions to the same problem

Obviously this must be an explicit process, where the tacit input is first encouraged, then amplified, assessed, amplified again until either the issue dies, is resolved, or mutates into another issue. But make no mistake; it’s the tacit input that drives the successful implementation.

Essentially we are making explicit that which was tacit; but on a selective basis, right time, right people, right place.

There is downside, however. Creating a workable tacit unstructured information BI system with the above features is non-trivial. I have done it many times, and it was never easy.

Caveats and Dependencies

Cultural Crevasses

Culture of collaboration is the all important enabler. If the people related barriers to sharing the knowledge creation process are not addressed, the venture will fail. No question about it. I have made an earlier post on the cultural issues and how they can be managed, but, briefly, the most critical barriers are, in my experience:

  • There’s no reward mechanism for contributing intelligence, and it’s a lot of work for no personal benefit
  • You don’t know who to tell, and it’s a lot of effort to find out
  • You don’t know if this BI snippet you have come across is accurate, you don’t want to bother someone else unnecessarily and someone else must know it anyway
  • There’s no important person around to hear what you have to say; so keep this intelligence to yourself until there is the right audience – the more valuable it is, the longer you’ll wait.
  • Tall poppies lose their heads, so keep your head down, and messengers get shot
  • You don’t want to embarrass your boss, or peer group, so keep it quiet

Source Validation

The source of intelligence is most people’s key to determining apparent accuracy of any tacit input. If you get a stock market tip, you will always check where it came from before acting. It’s the same for a rumor on a competitor’s product recall.

Audience Creation

Dissemination is completely dependent on adequate categorization. If a document, email, news item, etc. is not classified it cannot be circulated to the right audience. And everyone in the business must use the same terms for categorization, or they will miss relevant documents.

Crucial Taxonomies

This implies a standard comprehensive corporate vocabulary or taxonomy. Setting this up is not trivial either.

Automatic Categorization is Oversold

It’s not sufficient to classify documents by internal content references. The real, useful keywords for document that is relevant to BI may not even appear in the text. In spite of the tremendous advances in text analysis, the personal categorization by a subject expert still wins the classification stakes, in my opinion. By all means use the automated technique to get the item to a subject expert, but he/she will always be the best determinant of cross-references, importance and time criticality.


I believe that an important principle BI analysts need to fully understand is “the strategic and most valuable information in your business is in the minds of the managers and professionals” as first enunciated by Henry Minzberg. Turning this tacit unstructured information into explicit useful stuff is universally a high priority task. Done well, it creates the difference between learning and non-learning enterprises.

Unstructured Information – Tacit Versus Explicit for Profit and BI Best Practice August 9, 2007

Posted by Cyril Brookes in General, Issues in building BI reporting systems, Tacit (soft) information for BI, Unstructured Information.
add a comment

A picture may be worth a thousand words, a news item also has about a thousand and a marketing strategic plan may have around five thousand. OK, but a great idea for a new marketing message, an expert’s adverse comment on the marketing plan, or a chance serendipitous airplane conversation about a competitor’s plans may be each worth a million dollars, for just a few hundred words. What do you want, words or dollars?

I believe there is far too much emphasis on the analysis of documented unstructured information as a BI resource. The basic important data just isn’t there for most businesses. You can search as long as you like; mine it, categorize it, summarize it, but to no avail, the well is dry.

This post follows on from my earlier, definitional, piece on this subject.

Of course, I recognize there is potential import in some written material, for example, recent emails, salesperson call reports, customer complaints and their ilk. But these are like seeds, rather than the fruit off the tree. They are the beginning of a BI story, not the whole enchilada.

At risk of making the discussion too deep, Dear Reader, I think we need to consider the basic concepts before coming to any conclusions about how a corporation should manage its unstructured data, and the tools required.

I find it valuable to characterize unstructured information with a 2 x 3 matrix.

The horizontal axis has the above two basic categories of unstructured information:

Explicit unstructured items are those that are basically unformatted, but have a physical, computable, presence; e.g. documents, pictures, emails, graphs, etc.

Tacit items are basically anything unformatted that is not explicit, they’re still in the minds of professionals and managers, but are nonetheless both real and vital; e.g. mental models, ideas, rumors, phone calls, opinions, verbal commentary, etc.

The vertical axis has the three categories of unstructured information (according to moi!): independent, qualification and reference items.

Independent items stand alone being self-explanatory in the first instance, not requiring reference to other pieces of information, be they structured, unstructured, explicit or tacit.

Qualification items have an adjectival quality, since they add value to other items (structured or unstructured), but are therefore relatively useless without reference to the appropriate one or more Independent or other Qualification items (note there may be one or more threads to a discussion based on an Independent item)

Reference items are pointers to subject experts who can provide details or opinions, and other sources of information, structured or unstructured, together with quality assessments of the value, reliability and timeliness of those sources. As Samuel Johnson said “Knowledge is of two kinds. We know a subject ourselves, or we know where we can find information on it”.

Here’s a descriptive tabulation.



Independent, stand alone items

Meeting minutes

News items

Analyst reports

Marketing call reports

Legal judgments


Government regulations

Suggestion box items

Customer complaints

Strategic plans

Manuals of best practice

Emails about new issues or competitive intelligence

Unrecorded meeting discussions


Suggestions (undocumented)

Potential problems


Competitive intelligence from informal customer/industry contacts

Stock market (racehorse) tip



Off-the-record talks with government officers

Qualification, commentary items

Written comments on a report/news/analyst item

Documented opinion on problem or situation

Formal assessments of status implications

Verbal comments on a report/news/analyst item

Verbal comments on emails

Verbal opinions on problems

Verbal assessments of issues

Possible solution options

Comments on a rumor

Reference, source quality items

Lists of subject experts

Ratings of experts

Document sources, catalogs

Written reviews of document sources

Unrecorded subject expert identity

Opinions on expert quality

People who “know-how”

Informal unrecorded information source documents

Assessments of document source utility

Ask yourself, Dear Reader, which of these cells contains high value information, likely to assist your corporate executives find problems and make decisions? If they’re only on the explicit side, then you’re in the sights of UIMA and lots of enthusiastic vendors; good luck. If some are on the tacit side, please read on.

I’ve covered several of the relevant aspects of managing tacit information in earlier posts, e.g. here and here. However, there are some additional relevant observations to be made in the tacit versus explicit context.

  • The first, possibly most important, observation is apparently self-defeating to my thesis. All important, currently relevant, items of tacit unstructured information should be made explicit as soon as practicable.
  • It is not possible to identify, collect, store, disseminate, and facilitate collaboration on purely tacit items; it will happen in a “same time” meeting, of course, but wider ramifications demand that the prelude and/or outcome be made explicit.
  • Independent intelligence items, be they initially explicit (e.g. a recent email) or tacit, are very rarely complete as regards background to the issue, its importance to the business, its time criticality, and assessments of potential impact. If you will, the knowledge has not yet been created, only its seed.
  • The information required to complete the knowledge building that starts with an Independent Item is rarely in one location or person’s mind.
  • The knowledge building is based mostly on tacit information
  • The knowledge building process is most effective if performed via collaboration between the people who have, or know where to find, the necessary Qualification Items of information.
  • Some process for collaboration audience selection is required, one based on issue content, criticality and importance. It shouldn’t be left to pure chance.
  • Desirably, the collaboration process, but certainly the end result, should be made explicit, to avoid resolving the same issue many times over.

In my previous post I offered some questions that might provoke your curiosity, Dear Reader

  1. What are the most useful sources of unstructured information in our business? Are they Explicit or Tacit?
  2. If Explicit, how do we best marshal the information and report it?
  3. If Tacit, ditto?
  4. Is the information we get from our unstructured sources complete, and ready for promulgation, or do we need to amplify or build on it before it’s useful?

I expect that you will be able to answer 1 and 4 for your business; I’ve outlined the issues as best I can.

I’ll defer offering pointers you might consider for 2 and 3 to the next post, because I believe we still need to revisit the processes and constraints that inhabit the strange corporate world of collaborative knowledge building.

Unstructured Information in BI – Only for Spooks? What about Business Analysts? July 30, 2007

Posted by Cyril Brookes in General, Tacit (soft) information for BI, Unstructured Information.

There’s a big marketing and consultant push on UIMA, unstructured information management architecture. But I think it is largely missing the point for the real world corporate BI people, i.e. those not spooks or librarians. The critical concept, ignored by many, is that unstructured information is of two kinds, explicit and tacit. Even Wikipedia gets it wrong, ignoring the latter.

I believe BI gains most from tacit intelligence, but that’s not where the product marketing thrust lies. The importance of tacit unstructured information in BI is summed up well by Timo Elliott in a cartoon and by James Taylor’s recent blog post.

The heavy hitters in BI software, as evidenced by the recent takeover activity, e.g. Business Objects and Inxight for one, are pressing home apparent advantage to be gained by corporations with analysis of masses of emails, news, documents, etc. See, for example, the description of UIMA.

Well and good. But you can’t make a silk purse out of a sow’s ear, as Jonathan Swift said. And you can’t create relevant action oriented information for executives out of data that has no embedded useful information in it. The ocean of documents, with some exceptions, is a BI desert for most companies. But I mix my metaphors.

Basically, I believe that a corporation’s vast compendium of historical documents has little BI relevance. It may be useful to track or assess a person’s background, or to isolate the cause of a problem. But history rarely contains the up-to-date information that’s relevant to managing a business, assessing current performance and finding problems. The real lies in exploiting the tacit stuff.

I’ve quoted Henry Minzberg often before, but it bears repeating, as the message hasn’t yet been fully understood in the mainstream of BI: “The strategic database of an organization is in the minds of its managers, not in the databases of its computers”. This is as true today as it was in 1974. Today, one can add: “Or in the morass of historical documents and emails”.

Of course recent emails and documents often contain important information that can, and should, be part of a BI context; but usually only as the seed for a collaborative knowledge building process. This is the nub of the issue; it’s hard to identify, collate, disseminate and collaborate on tacit unstructured information. Perhaps this is why most authors steer clear of the issue. But we need to address it if we are to be effective. More on this in my next post.

I wrote last year detailing some of my research in the 90s on the subject of “hard” and “soft” information, how valuable it is in many BI contexts particularly CRM, but also how difficult it is to exploit. In this context hard information refers to the structured, numeric, formatted, BI reports. Soft information is the unformatted, unarticulated, information in managers’ and professionals’ minds.

An interesting article by Rick Taylor deals with unstructured information is relevant here. It says, in part:

The key to defining knowledge management is to make sure you are separating “explicit” knowledge from “tacit” knowledge. Explicit knowledge is anything easy to quantify, write down, document or explain. Tacit knowledge is everything else. The knowledge based on ones experiences, and often times, at a subconscious level. It is information that you don’t necessarily know you know until you are reminded of it. If you were asked to write down everything you know, could you do it?

The explicit and tacit labels were used first in this context, I believe, by Nonaka and Takeuchi in The Knowledge-Creating Company.

The BI key questions that arise from this discussion are, I believe:

  1. What are the most useful sources of unstructured information in our business? Explicit or Tacit?
  2. If Explicit, how do we best marshal the information and report it?
  3. If Tacit, ditto?
  4. Is the information we get from our unstructured sources complete, and ready for promulgation, or do we need to amplify or build on it before it’s useful?

I believe that the above analysis outlines the problem of utilizing tacit unstructured information reasonably well. I’ll offer my answers to these issues in the next post.


Social Bookmarks and Tagging in BI Fail the Just-in-Time Test February 20, 2007

Posted by Cyril Brookes in General, Issues in building BI reporting systems, Tacit (soft) information for BI, Taxonomies, Tags, Corporate Vocabularies.

Tagging and Social Book-marking for BI applications is a hot topic. See, for example, Bill Ives comment. But I think there are barriers to it’s success in the corporate context. It doesn’t lend itself easily to the dynamics that are, or should be, key aspects of BI system design.

Sure, I am completely in agreement that information, particularly soft information, needs to be tagged, or classified, before it can be useful. I’ve talked about this several times in this blog. Social book-marking is better than none.

If information isn’t categorized then it cannot be selectively disseminate or easily searched for.

The social book-marking ethos implies that people create their own tags. But, of course, no one else knows (at least knows in a short time frame) that this tag is being applied for this purpose.

Until the tag’s existence and meaning is widely known, no item of, say, competitive intelligence with this tag can be subject to targeted personalization to relevant decision makers. More importantly, if the tag describes a concept that is identical to, or nearly so, those linked to one or more other tags then confusion is likely.

It follows that social book-marking can be effective in information retrieval, if the tags are managed, moderated and disseminated. However, this approach is not likely to be valuable for alerting purposes, especially in dynamic business environments. This is because those being alerted will not know of the tags existence, and will be frustrated by multiple tags with the same meaning.

In any case, corporate wide management of social bookmark tags is always going to be a big ask.

Knowledge in a business is often created via group collaboration. The smart corporation enables such new knowledge to be disseminated rapidly to those who should know it, and can take requisite action. There is no time to create new tags that may be redundant anyway, and to disseminate their existence and meaning widely.

Business intelligence has two basic purposes:

1. Helping executives and professionals assess status and find problems

2. Supporting problem solving, usually by less senior staff

For the corporate BI context the alerting and problem finding objectives are usually more valuable than problem solving. Knowing an issue exists will often be absolutely critical, resolving it is usually less difficult and less important. We cannot solve problems we don’t know exist.

As I opined recently, it is the combination of subject matter and assessed importance that is the key to effective alerting, or selective dissemination. And if an executive is to have a personalization profile it must use tags that are pre-specified and whose meaning is understood widely. Social book-marking does not usually imply assessing importance. Often importance can only be determined by people outside the group that creates the information, and the tag.

In the BI context a corporate vocabulary of preferred terms will be more useful than various sets of personally created, and probably redundant, social bookmarks. This is because the standard terms are widely known. Further, they are usually grouped in hierarchies of broader and narrower concepts and this facilitates retrieval and alerting.


Executives can seek items of high importance that are classified by a broader term (say, overall gross margin issues), or those about a narrower term (say, product X gross margin) that are of lower importance. In either case, they will not be inundated with large numbers of items.

Of course, inside a project team and other tightly knit groups social bookmarks may be suitable ways to tag documents and other material for retrieval.

However, I don’t believe that the wider corporate environment will benefit to the same extent. It’s a case where more formality and discipline brings better results.

Collaborative BI Implies a Personalized Grapevine – but, make it Smart Alerting or its all Blah! February 12, 2007

Posted by Cyril Brookes in BI Requirements Definition, General, Issues in building BI reporting systems, Tacit (soft) information for BI.

Effective collaboration depends on the dynamic creation of groups that can exchange and share intelligence. Collaborating people in a group create knowledge; often it is new knowledge that can improve business performance. However, finding the right group participants, and disseminating the knowledge to empower action, both require targeted, selective dissemination, of information – that’s personalization. Truly, I heard it on the grapevine!

Some regard personalized alerting in BI as creating a “market of one” for information..

I disagree. As I see it, it is creating a group of relevant people, the “A list” if you will, for the issue at hand. How can this happen, not occasionally with serendipity, but routinely? Groups must be dynamic, different for each issue, expanding and contracting in size as the issue grows in importance, or declines.

Markets of one work for marketing situations, e.g. books with Amazon.com, but I don’t believe it is the paradigm for collaborative BI.

Clearly, the traditional BI report, with information prepared by others submitted to potential decision makers is discredited. Today we have lakes and lakes of information available; Herbert Simon got it right in 1971: “Information abundance creates scarcity of attention”. And one can add: Knowledge poverty.

Informing decision makers doesn’t cut it anymore. Maybe it never did? We need to change the process, introducing dynamics to the grapevine.

Issues grow in business importance when people, in the know, determine they have grown in importance. There’s no other way.

All messages, ideas, news items, etc. on a topic are not of the same value or criticality to a business. Most are irrelevant to decision makers; they are waffle, padding, dross, blah.

Some of those items will be interesting to the professional; fewer are important, business-wise; but very few are critical to the business. How do we distinguish? Well, it’s simple: subject experts tell us they’re critical.

If you’re still with me, Dear Reader, personalized alerting, selective dissemination, of intelligence items on a topic can only be effective, therefore, if someone tells us (or the dissemination authority/process) what is important and what is not.

I don’t believe that automated importance classification works in practice – in a business anyway. It might do for spooks, but not the rest of us.

Some years ago, I built a selective dissemination collaboration system based on a patented importance escalation process. I called it grapeVINE. It employed this model of escalation and dynamic audiences for information. It was most effective when seeded with news, marketing reports, or other items. They were automatically classified, using a standard taxonomy or vocabulary, and selectively disseminated based on client interest profiles.

grapeVINE’s special character emerged when a subject expert commented on an item, raising it’s importance level – saying something like “this is important because the implications are….”. Immediately the audience would increase for this, and only this, discussion thread. More people are interested in important stuff than dross. One of these new recipients might then escalate the discussion further, bringing in more people – likely action oriented players. Then the game is on.

Two dimensional personalization of business intelligence, based on a combination of subject matter and importance to the business, is an effective driver of dynamic group formation.

Provided the culture of sharing is established in the business (and that’s an important IF), the potential for improvement in decision making is immense. It is the optimal vehicle for combining structured (numeric) and unstructured (text) information into BI systems.

Paraphrasing Crocodile Dundee: THIS IS A GRAPVINE!

Personalization in BI: Selective Dissemination and Targeted Retrieval of Important Information January 30, 2007

Posted by Cyril Brookes in General, Issues in building BI reporting systems, Stafford Beer, Tacit (soft) information for BI.
add a comment

Personalization in BI grows in significance with the near universal recognition that passive reporting, designed for the masses by supposed experts, is limited in its utility. Action oriented reporting is preferable; it always has been. However, many business analysts do not recognize that selective dissemination of information, aka personalization, is a pre-requisite if reporting is to stimulate action. Only specific people can or need to take action, and common sense tells us that they must be targeted.

See, for example, the article by Neil Raden.

Two sorts of personalization apply in a BI context:

         Push and Pull

Each can be applied to external or internal recipients.

The focus of this blog is on selectively pulling information, predominately by internal people. This is the principal aim of action oriented BI, directing valuable information (and only valuable information) to executives and professionals that assess a situation and/or take action as a result. This is not to say that other aspects of BI, such as keeping people informed as to status of the business, should be ignored. But these objectives are far less vital than supporting executvie actions.

I leave discussion on the much more fraught selective information push situation to others. Determining what information will be of interest to a customer or supplier and pushing this category of stuff to them can be a valuable marketing tool, or (more likely IMHO) a PR disaster. We always hear of Amazon.com and its success with cognate book promotions, but books are easily categorized in a universally accepted manner; most other items are not so easily classified , and the implications for inappropriate information push can be dysfunctional.

Any discussion on the effectiveness of BI for improving the quality of executive decisions (and what other purpose might it have?) must have regard to the actual decision making process. The theory of this process is well established, notably by Herbert Simon. Many researchers have also considered the relationship between this process and the information required for its operation. In this context I particularly value the work of Stafford Beer and Henry Minzberg.

Information that enables effective decision making belongs to one of two categories, and both are essential if decisions are to be optimal:

It helps the executive find problems and opportunities – situations that need a response.

Stafford Beer calls this Attenuation information, and I have discussed this in detail earlier

It helps the executive solve problems he/she has found (or been told about)

Beer calls this Amplification information, also discussed in this blog earlier

But I diverge.

Returning to the personalization theme; selective dissemination is vital in the problem finding context.

Obvious candidates include:

  • Alerting the executive to important exceptions, out-of-specification performance, unusual situations, adverse forecasts of key indicators, and unacceptable (or advantageous) trends.

  • Equally, if not more, critical is soft information (opinions, comments, assessments, etc. that portend problems, or throw doubt on the accuracy of factual information.

Targeted information retrieval is also vital to support problem solving.

  • The solution process that needs supporting includes diagnosis of the severity of the problem (what will happen if nothing is done), identifying possible alternatives and assessing their implications.
  • During a decision making process executives must be able to retrieve important, valuable, information as distinct from the routine stuff. This applies to both factual and soft (tacit) information. In this context, the latter includes ideas about problem implications, suggestions for potential solution alternatives and recollections about what we did last time this happened.

The key word in both these situations is “importance”.

Alerting to, and targeted retrieval of, useful information implies that some assessment must be made of the significance of a data item, either using an automated rule system, or a personal assessment.

Truly this is the stuff of business intelligence. Without importance classification all information is equal, but obviously this is not reality.

Selective dissemination and targeted retrieval, the basis of all personalization, depend therefore on the BI context being able to distinguish information importance as well as its subject, topic, or data class.

Importance, in turn, depends on two characteristics: urgency and value to the business.

I have experimented over 20 years with different retrieval/alerting procedures for corporate BI systems, using both automated and human importance assessment. I’ll detail this experience in the next post.

BI System Design incorporating Wiki and other Web 2.0 Components January 10, 2007

Posted by Cyril Brookes in General, Issues in building BI reporting systems, Tacit (soft) information for BI.

Suddenly collaboration is flavor of the month, or year anyway. Customers are re-designing products, buyers are guiding the choice of other buyers, repair and service people are specifying work procedures, those with spare time collaborate in wikis-everything; and maybe the lunatics are running the asylum? But what of Business Intelligence systems, what is their place in all this?

I’ve been preaching the utility of collaboration an essential element of BI for 20 years now. Maybe it’s going to happen at last?

But, even with the current enthusiasm, it won’t happen at Youtube speed. Collaborative BI is more complex than just loading some videos or other data into a category for others to retrieve. And, supposedly, corporate people don’t have time to surf the Intranet, let alone the Web, looking for relevant stuff.

Therefore, it will take time; and remember the old timer’s adage: “You can tell the pioneers; they’re the ones with the arrows in their backs!”

Here’s a set of axioms that I believe are relevant if we are to succeed in this collaborative endeavor, all of which raise barriers, some large, some not, it depends on the business environment:

  • Corporate people who come across Web 2.0 style intelligence often don’t know its value, and whom to tell
  • They usually only have part of the story anyway
  • They often lack the background to be able to assess implications
  • Supplying intelligence to Web 2.0 style repositories or applications is time consuming, and may not be at all rewarding to the author, only to others
  • Intelligence can’t be searched for, or be subject to push messaging and alerting, if it’s not categorized
  • Categorization must conform to a corporate standard vocabulary, or it will not facilitate sharing and collaboration
  • All BI items indexed by a category are not of equal significance or value; some may be critical to the business, others routine news that’s already well known
  • The high value items ought be separated from the dross and given wider audience, or personalization will be ineffective; but how to do this?
  • BI is useless unless the recipient can assess its implications, and often this requires additional input of BI, or experience, from other people or sources – the collaboration imperative
  • Corporate collaboration raises infinitely more cultural and behavioral red-flags than Web 2.0 practitioners could dream of; see earlier post.

Nonetheless, I’m sure that we’ll see increasingly effective means for accommodating the issues raised by the above points. I have suggested some design principles in another earlier post, but the rapid evolution of wiki style knowledge creation, with the attendant blog explosion, is opening up new opportunities.

I believe that the issue of bringing new knowledge to the attention of the right people, personalized distribution as the knowledge is created, will remain a substantial BI issue. My ideas on this will be the subject of a later post.

Knowledge is Power – But Only Sometimes in BI December 1, 2006

Posted by Cyril Brookes in General, Tacit (soft) information for BI.
1 comment so far

The management, or exchange, of soft (tacit) information (knowledge) is a current focal point in the BI space and we hear the ubiquitous quote frequently.  Unfortunately, knowledge isn’t always power, in fact it’s rather like Francis Bacon’s original Latin version:  Nam et ipsa scientia potestas est (my favored translation is: For knowledge, too, is itself a power).  

The indefinite article makes all the difference.   

Two interesting papers on the Knowledge is BI Power theme I’ve read lately are: Do we need internal knowledge markets?  by Jean Graef of the Montague Institute and  Making a market in knowledge is Lowell Bryan’s (McKinsey & Co) contribution.  Graef summarizes the Bryan article as follows: 

According to  Bryan, most companies have tried one of three approaches to managing knowledge with “mixed success.” (1) Technology alone doesn’t work because most corporate documents are out of date, poorly written, and can’t be easily “parsed” (read) by computer software. (2) Content “pushed” to employees by a centralized staff of communicators doesn’t work because “it’s not very valuable to most frontline employees.” (3) Allowing each business unit the freedom to manage its own knowledge works “because it facilitates exchange among small groups of workers with common interests,” but it “provides just a fraction of the potential benefits of exchanging knowledge on a company-wide scale.” 

Bryan’s solution is an internal knowledge market, which he defines as “the exchange of items of value among parties who don’t know each other.” 

Key qualifiers of knowledge’s potency in the BI context include, in my experience: 

  • Not all knowledge has value 
  • Knowledge’s value is ephemeral 
  • Most valuable soft knowledge is tacit 
  • Knowledge must be explicit to be useful in BI 
  • Knowledge is often spread over several minds 
  • Just-in-time collaboration is needed to create required knowledge 
  • If knowledge is not categorized it cannot be shared 
  • Categorization must be according to a vocabulary (taxonomy) of standard terms 
  • Context impacts both value and implications of knowledge 

Importantly, people who want to know something don’t know if it exists, and those who know it don’t know if it is useful, or to whom.  Even if the knowledge does exist in some useable explicit form, it often requires some facilitator to bring its relevance to the people who need it.  Some form of context dependent information push is therefore desirable. 

The central thrust of my argument is that many commentators appear to assume that the knowledge to be shared, managed, marketed, exchanged or whatever, actually exists in tangible form.  In my experience, the most valuable knowledge is most often created through collaboration between two or more minds at the time it is required to be utilized.   

Authors can be encouraged and motivated to make explicit their tacit knowledge;  if they know they know it.   Editors can review and collate knowledge that is already explicit. 

 It is my view that the real issue to be resolved in the business enterprise is the stimulation of knowledge creation as and when it’s required, say in response to a new problem or the resurfacing of an old problem.   

I don’t believe that a market approach is sufficient, but definitely incentives are required.  These ought include corporate recognition awards that counteract the natural cultural barriers that limit collaborative inclination among staff. 

A second theme of mine is the need to recognize that there are three types of BI related knowledge available, or being created, in a business enterprise (Independent; Qualification and Reference).  They have different power attributes, and require different treatment in the sharing context.  I’ve discussed these in some detail in an earlier post. 

To summarize my opinion on how to maximize knowledge collaboration: 

  • Stimulation via news and industry related feeds is a critical starting point for commentary that leads to knowledge creation 
  • Subject experts who add value are key elements of the collaboration scene; they separate the value from the dross 
  • Escalation of the more important discussion threads, to a wider readership, is necessary since not all items have same value to the enterprise 
  • The reward mechanism is critical, as discussed 
  • Work-shopping of intelligence ought be facilitated to overcome the natural reluctance of people to share information they are not sure about with those they do not know. 
  • Categorization of knowledge items is essential, and should be done according to a vocabulary of standard terms. 
  • Knowledge is certainly one power in BI , but not always.

Managing Cultural Barriers to Effective Collaboration; Enabling Action Oriented BI for CRM and Competitive Intelligence October 26, 2006

Posted by Cyril Brookes in BI Requirements Definition, General, Tacit (soft) information for BI.

In previous blog posts I’ve opined on the cultural barriers impeding knowledge sharing and limiting BI effectiveness in these and other application areas where decision making depends on information about the future probabilities, rather than historical fact.  I proposed a “Top Ten” set of reasons why routine collaboration on these issues is difficult to instill in the professional employee’s mind.   

I also outlined my assessment of the nature of the soft, or tacit, information resources that contain the intelligence, or perhaps the knowledge, that must be marshaled into the BI context if a successful CRM, CI or other subjective decision support system is to be created. 

For completeness, these top ten and categorizations are reproduced here: 

Culture barriers that compromise the collaboration instinct: 

  1. You (not actually you Dear Reader, but those other unwashed professionals!) don’t want to start a debate you cannot control as it evolves 
  2. Tall poppies lose their heads, so keep your head down 
  3. Messengers, especially whistle blowing ones, get shot – or have short careers, so be silent 
  4. There’s no important person around to hear what you have to say; so keep this intelligence to yourself until there is the right audience – the more valuable it is, the longer you’ll wait. 
  5. You don’t want to embarrass your boss, or peer group, so keep it quiet 
  6. You’re not sure that the intelligence is correct, so just in case, keep it to yourself so you don’t lose “face” 
  7. You’re reluctant to communicate with people you do not know 
  8. You don’t know who to tell, and it’s a lot of effort to find out 
  9. There’s no reward mechanism for contributing intelligence, and it’s a lot of work for no personal benefit 
  10. Your boss will only steal the idea, so don’t even think of telling 

I know of three categories of soft (tacit) information in my experience.  These form the basis of my soft information metadata categorizations: 

  1. Independent Items: stand alone intelligence, e.g. rumors, ideas, industry gossip, leaks from competitor or customer sources, etc. 
  2. Comment, or Qualification, Items: e.g. assessments of a competitor’s new promotion, comments on sales forecasts, qualifications of opinions on a prospect list, etc. 
  3. Reference Items: Lists, with quality assessments, of subject experts or other resources that can supply details or opinions on hard or soft data or events. These cultural barriers and intelligence categories summarize the problem.  They also hold the key to effective synthesis of BI design and management strategies that will ameliorate the situation adequately. 

What, then, are the guidelines leading to effective synthesis that mitigates the culture issues?  I divide them into three categories: 

  1. Collaboration rules set by the enterprise 
  2. Subject Expert panels and their operation 
  3. Code of “good collaboration practice”  

Setting the corporate agenda for collaboration

This is a set of suggested “not-negotiable” rules for establishing instant messaging, chat, different time/different place conferencing, etc. in the enterprise. They are based on my experience with several hundred installations.  Of course, they may not be relevant to your business.  If I sound dogmatic it is because I have seen so many failures that could have been avoided. 

  • Categorization, Nazi style:  Information that is not categorized cannot be shared effectively in any group over 20 people.  Automatic categorization is probably essential for any business unit of reasonable size.  Categories used should be drawn from a restricted vocabulary, and synonyms should be banned, or actively discouraged.  See my earlier post on Vocabularies for sharing tacit information of September 14. 
  • Importance Qualifiers:  Subject based categorization is insufficient by itself for effective content based dissemination.  There are simply too many items on most subjects.  A “value to the business” indicator is also required that can be used in search queries and personalization profiles.  Most professionals want to receive important items on a subject, possibly even all high importance items irrespective of subject matter,. 
  • Rewards for Added Value:  It takes time and effort for a professional to submit tacit information, especially when it’s related to CRM or CI.  Plus there is all the downside potential listed in the above “top ten”.  There must be a reward system in place to encourage participation.  This requires regular monitoring of the system, and may include public acknowledgement of valuable contributions.  The options for rewards are many, and I won’t dwell on them here. 
  • Seed Items:  The most laborious task in contributing soft business intelligence is introducing the context.  I have found that often using news feeds and other automatic sources can stimulate the contribution of ideas and assist in recalling intelligence items. 
  • No Anonymous Items:  Intelligence must be owned by the original contributor.  This is essential to limit submission of garbage, provide rebuttal opportunity if needed, and especially to assist with assessment of the value of the contribution.  The author’s identity is often the best guide of value and importance to a reader.   
  • Use Subject Experts:  This is a critical element.  These specialists are essential if the “wheat” is to be separated from the “chaff”.  Their role is discussed in more detail below. 
  • Facilitate Work-shopping of Ideas:  Many professionals are reluctant to expose half-formed ideas to an enterprise-wide audience.   It should be easy to form dynamic, short life, or long life, workgroups that can refine a proposal before it is released to the “world”. 

Subject Expert Panels 

Filtering of soft information is essential to enable appropriate impact and action assessments.  On first receipt of an item, of whatever type, subject analysis is probably the only useful action.  I have little confidence in the automatic importance assessment tools available, but maybe I’m old fashioned.  They may be appropriate in slowly evolving contexts, but not in most businesses. 

I believe that only a subject expert can consistently validate the importance and urgency of a raw piece of intelligence, especially in the CRM and CI environments which are so dynamic.  

Suggested Subject Expert Guidelines: 

  • More is better; experts should only be required to cover a few narrow topics.  More than one expert per subject is usually beneficial if the topic is important enough. 
  • Avoid self-appointed experts; there should be a formal approval process 
  • Experts should be empowered, and encouraged, to add value with text as well as escalating value/importance and adding new keywords if appropriate.  Often the original keyword categorization will not be appropriate, or there will be cross enterprise implications only apparent to the expert. 

Code of Good Practice 

Guidelines for collaboration behavior are obviously desirable.  Suggestions include: 

  • Reward and endorse good behavior, ignore bad behavior.  “Flaming” generates ill-will and ought to be discouraged.  Judicious use of importance escalation is usually effective, and people rarely read routine low importance items on a subject. 
  • There should be no adverse reaction to items submitted at routine low importance on any subject.  The relevant Subject Expert will be able to assess the utility or otherwise of such an item, and escalate its distribution if appropriate, or let it die otherwise.   
  • Obviously repetitive dysfunctional behavior should be dealt with.  But this includes efforts to “shoot messengers”. 
  • Knowledge creation through idea building should be encouraged, rather than creation of new independent items.  Piggy-backing of ideas is important.  Retention of original ownership of ideas is important, to encourage participation.  If someone adds value to an idea, they should get value for the addition, but not the original concept. 

There are several collaboration packages available that facilitate implementation of these guidelines, for example that offered by Sun Microsystems Inc.  The hard part is setting up the environment, not choosing the package!  

That’s about all I can offer on this topic – good luck!