A recent article in the McKinsey Quarterly, Rethinking knowledge work: a strategic approach by Thomas H. Davenport, struck a chord with me; an old, familiar chord.

Mr. Davenport has done a fine job of delineating the two extremes of the continuum from unstructured information (textual or document-oriented) to structured data (relational or transactional) and the styles of information-processing applications that best suit these two distinct types of information. He has also done a valuable job of describing some of the implications, good and bad, for the efficiency and productivity of knowledge work.

In particular, the decision matrix in the article exhibit is a compact and useful addition to our toolbox for responding to productivity problems related to the efficiency of support that IT solutions provide for business processes.

The dimension of structure in information management is hardly a new concept. Sadly, I am old enough to remember when nearly all IT applications were exclusively concerned with structured, transactional data. These applications revolved around accounting, finance, payroll, taxation etc. In fact, in those days there was no IT as such; we called the whole field "data processing".

How quaint all this sounds now. In 1992, I got wind of a daring new product named Lotus Agenda. This was something like a spreadsheet, but you didn't have to structure your information in order to store and retrieve it. Agenda would accept bucket loads of plain old text: documents, paragraphs, quotes, prose and poetry. You didn't have to have any predetermined knowledge of what the information was, you just shovelled it in. Once it was in there, you just queried your pile of text and Agenda would find instances of search words. It would also find synonyms, antonyms, relationships and threads, it would discover the most used terms or phrases, and it could also construct virtual folders based on your search terms.

Take one guess: what does that sound like?

Full marks if you guessed "Google" or "search engine". Nine out of ten if you answered "the Internet".

Lotus Agenda was ahead of its time, and it was a solution in search of a problem. It didn't make the cut. But it was one of the first serious attempts to grapple with the content of unstructured information. The tool and its operation was different because the nature of the information was different: it lay at the opposite extreme of the dimension of structure.

Mr Davenport's article made me reflect on my own development efforts over the years and the shape of the product range I've finally come up with. This thumbnail from one of my posts captures the point:


Growth path 600


As you traverse this product range from bottom to top, you are also traversing the continuum of structure, from:

  • Unstructured content in CMSs
  • Semi-structured content in eCommerce and CRM systems
  • Structured information in an ERP system
  • Completely structured, transactional messages in an EAI system

Observe also that at the same time you are traversing a continuum of value, from:

  • Commodity CMS, eCommerce and CRM systems
  • Commercial, business-critical ERP systems
  • High-value EAI solutions that can multiply value across an extended supply chain (when properly applied).

This makes a kind of notional sense. Unstructured information abounds. It's a commodity, so it seems appropriate that systems that handle unstructured content are commodities. By contrast, the value and richness of structured information derives largely from its structure, so it seems right that structured systems are specialised to their domains, there are fewer systems that fit any given domain, they are harder to build and support, and they command a correspondingly greater investment.

Also, unstructured information generally has little value until a human reads and comprehends it, perhaps synthesising it with other related information, and then makes a decision or takes an action based on the assembled information. All this represents an overhead cost in realising the value of unstructured information.

In contrast, machines can generally perform actions based on structured data (depending on the quality and context of the data), with minimal or no human intervention. This represents a reduction in the cost of extracting value from the data, and possibly a corresponding increase in the perceived value of the data.

But as Mr. Davenport pointed out, there's a fly in the ointment. Humans don't respond well to automated recommendations, decisions or directions. I have learned to expect that there will inevitably be resistance to the introduction of highly structured automation systems, and I share Mr. Davenport's concern that this natural trepidation and defensiveness has to be handled with great care and sensitivity.

Up to top of page