Over the last several years I’ve been in the midst of helping companies build intelligent content applications using taxonomy, and almost all of these had already spend a pile of cash on search and content management technology, only to see it fall short of their vision. Many had even implemented component content in DITA or another XML vocabulary. Taxonomy helps to bridge the gap in several important ways.

First of all, taxonomy helps companies organize and manage their source content. A well designed taxonomy can become the basis for a metadata and content type strategy for the CMS, and the source of the controlled vocabularies that content authors and publishers use to classify their content. Content classification is important for defining the “aboutness” of content as well as administering it. We all need well-defined, clear, unambiguous terms for administrative metadata, including our organization structures, customers, products, information types, and information security classifications, to name a few important categories. Leaving CMS users to enter this metadata by freely typing exposes us to human errors and inconsistency. At a minimum, we want to maintain an authoritative term list and expose it in drop-down lists for users to select values when they upload content to the CMS. Users are usually able to enter this metadata with low error rates when selecting from controlled term lists, especially if their job is content publishing. The list just makes it easy and consistent.

For “aboutness” metadata, we need to help users be comprehensive and consistent, so we often use the taxonomy to inform a classification engine that analyzes the document and provides suggested metadata to the user. Subject classification schemes are conceptually similar to the subject headings in library card catalog systems – they start with broad domains and categories that are broken into increasingly narrower topic spaces. The big difference is that each organization will need to develop and maintain subject classifications that are relevant to their business content. For example, I’m helping a high tech manufacturer classify technical documents; their taxonomy covers technologies, manufacturing process steps, customer needs / applications, as well as symptoms, fault codes, and root causes for troubleshooting and repairing their products. They had a rich set of terms for all of these topics, some in a corporate taxonomy and others in specific systems for service and quality management.  Putting them into a taxonomy helped us use that information for auto-classification of content. Metadata is proposed to content publishers when they upload it to the CMS, and they can add or remove terms form the proposed metadata using the same taxonomy the classifier used. The result is that documents are now more completely tagged with “aboutness” metadata in the CMS.

Intelligent content doesn’t end with the CMS, however. Search engines classify content using their algorithms to match end-user search queries (what you type in the search box) to content – whether it’s unstructured document content or structured data in an enterprise system, or both. A search engine doesn’t understand your business – it only looks at all of your content as a “bag of words” that it statistically determines to be “about” something looking at unusual combinations of terms,  or high-frequency terms. A taxonomy can tell the search indexing engine that certain terms are more meaningful to your business, and that there are relationships between terms that matter when it comes to relevance.  Even if the search engine is placing higher value on metadata values, those usually contain “preferred” terms – the official business labels. Users, on the other hand, are not so disciplined when they type in the search box – they may use “non-preferred” terms. A favorite example is searching a NASA site for “moon buggy” when NASA calls the item the “lunar excursion vehicle.” The taxonomy can relate those terms so the search engine returns relevant documents – even if they never contain the term “moon buggy” or that referred to the acronym for it.  

Finally, taxonomy can be used to driving the search user experience in major ways. It can become the basis for the facets in search refinement, allowing users to narrow their search along the dimensions of the taxonomy (show me only information about these document types, or these products, etc.). It can define the terms we show in tag clouds and other interface objects. The taxonomy can also help the search engine identify related searches – for instance, all of the astronauts who ever drove a LEV, or the Apollo missions that included a LEV.

I’ve actually seen a recent example of an intelligent content application that is entirely defined in a taxonomy – the UK National Health Service has build a Flash application that lets you navigate a hyperbolic tree of symptoms and diseases, all of which is directly managed in a taxonomy and flowed directly into the portal. Their taxonomy aids search and results relevancy as well by taking terms like “AIDS” and assuring that search engine stemming doesn’t return documents that contain the terms “aid” or “aiding” – imagine all the results for “first aid”, “band aid”, “hearing aid”, or “health aid”.  

We also tend not to think of intelligent content systems as being driven by rather “dumb” legacy content, instead they are all about XML and structured content. In fact, most of the intelligent content portals I’ve worked on in the last several years were being populated by legacy PDF content – which was made intelligent only through the use of auto-classification with a well-crafted taxonomy and exposed through faceted search. For lengthy documents, document preview technologies can help hone-in on relevant pages – also being guided by the taxonomy to map search queries to preferred terms.

My conclusion in working with search and enterprise content management technology is that taxonomy development and management is a key success factor in creating effective intelligent content systems. Taxonomy can inform content types and metadata schema, make for consistent tagging, harmonize disparate structured data, and drive dynamic search and navigation user experiences, even with not-so-intelligent content.

 I’ll be presenting more on this topic with extensive examples at Intelligent Content 2011 in Palm Springs, February 16-18. Hope to see you there!

When businesses implement intelligent content, they usually adopt a “day forward” strategy that assures all new content is “intelligent” (i.e., is developed in XML and dynamically published), and they minimize the volume of legacy content to convert and migrate. Semiconductor equipment manufacturers – like many other capital equipment manufacturers – support products that last 30 years or more, so legacy technical content is critical to keeping that equipment up, running, and profitable for their customers.

In one such company today, that legacy content exists as monolithic manuals in PDF format that are hundreds of pages long, or as PDF renditions of engineering drawings, or as data in enterprise systems in relational databases or ERP systems. Field Service Engineers spend many hours per week searching for content across multiple systems – ERP data, content repositories, engineering websites, drawing repositories, knowledge bases, technical forums, email, personal notes – to find the procedures, drawings, reference information, and expert advice they need to effectively troubleshoot and repair customer equipment.

In the future, that information needs to be seamlessly integrated into a single-point of access that provides the Field Service Engineer with information that is relevant to their current context: the product they are working on, the customer account, the current configuration, the current problem or fault condition, the latest engineering information and best known methods – all without entering a word into a search box.

The challenge for intelligent content is simply stated: How do we get there from here?

In this presentation at the Intelligent Content conference, we’ll discuss this company’s unique pathway forward to intelligent content:

• The complexity and richness of content types and sources that must be unified through search and navigation for the end user (service engineers);
• Building a firm foundation with a sound information architecture including taxonomy and metadata;
• Taking the first steps with a modular content strategy based upon PDF documents in SharePoint, with a unified custom search experience;
• Transitioning in later phases of the project to intelligent XML content integrated with enterprise data in a seamless, task-focused interface.

Thursday October 1, 2009

1:00 – 2:30 p.m. EDT

Register here for free.

In this session, we will hear a case study from the Hartford Insurance Group about how taxonomy was applied to improving findability of Underwriting content.  We’ll also learn about the state of the insurance industry with regard to adoption of metadata standards and taxonomy practices. This is based on our research into how insurance companies are addressing challenges around content and document management efficiency and search effectiveness by applying taxonomies and metadata.

Join us to hear Seth Maislin, Senior Consultant at Earley & Associates, and Jeff Auker, Assistant Vice President at the insurance company The Hartford, discuss an enterprise taxonomy initiative for managing over 200 million documents in support of underwriting and claims processing. They will address best practices for organizing a taxonomy project, methods for gaining support from stakeholders, and operationalizing a taxonomy program for ongoing success.

Paul Wlodarczyk, Director of Solutions Consulting for Earley & Associates, will review findings of our benchmark metadata survey updated with new results specifically for the insurance industry. The goal of the study is to measure industry progress on a Metadata Maturity Model scale, a road map designed to help get practitioners started – and continuously improve – content tagging within organizations.

Topics include:

• Applying taxonomy to improving findability of information
• Case study – The Hartford
• The role of metadata and taxonomies in content reuse
• Trends in metadata maturity for the insurance industry

Register here for free.

Speakers
Robert Berry, Information Developer, IBM
Michael Harris, Information Architect, IBM
Erik Hennum, Information Model Architect, IBM
Paul Wlodarczyk, Director Solutions Consulting, Earley & Associates

Many organizations have turned to component-oriented content to create more sophisticated knowledge products, in more languages, at lower cost. For most organizations these days, component content is achieved by using DITA, the Darwin Information Typing Architecture. Finding content in your file system or content repository is hard enough when you’ve got simple text documents to deal with. When you’re using DITA and other component-oriented methods, you increase the difficulty by two or three orders of magnitude, because you’re looking for smaller needles in bigger haystacks. It’s logical that DITA users would turn to taxonomy and metadata to improve findability of their reusable content.

Our research confirms that organizations that use XML authoring are more mature than their peers with respect to the adoption of best practices for search and metadata. However, the use of native DITA metadata capabilities is rare, and many are also missing out on opportunities to use taxonomy for reuse and improved findability. We will explore the metadata capabilities within DITA and component content management systems, discuss two major benefits that can be achieved by using descriptive metadata and taxonomy, and recommend some best practices for getting started with metadata for component-oriented content.

Check out the preview on SlideShare:

August 05, 2009
1:00 PM EDT

An enterprise search audit will determine where you need to specifically improve search processes.  Based on actual user behavior and system results, a search audit will provide hard data for a baseline evaluation of search effectiveness.  We’ll discuss types of search audits, approaches for gathering data and ways a search audit can pinpoint the needs for improvement  in system tuning, metadata and content management processes.

Jeannine Bartlett will present Strategies for Search Auditing

What’s in a search audit, and why should you consider conducting one? We’ll look at enterprise search security audits, a framework for measuring the enterprise search “experience”, and an approach to website search SWOT analysis. We’ll wrap up with a range of ideas for interpreting and leveraging search audit results.

Avi Rappoport will present Auditing Search: The Invisible Part

The searchable index itself doesn’t get much attention, but there are very important things there (or not in there) that define the quality of the search.  Search log top query and click-through metrics can indicate missing sources and other problems.  The index may need to update more frequently, it may have security and access control problems, it may be tuned for smaller disk space requirements, and the document store may be insufficient. Essentially a Search Audit should also analyze what’s in the index, what’s not, and why.

Price: $50.00

Register here

Date: Wednesday, July 15, 2009
Time: 1:00 – 2:00 Eastern Time
Cost: $50 (Survey respondents need not register and will receive an invitation to this call at no cost)

Register here.

Four years ago, a benchmark survey was conducted to understand how organizations understood and applied metadata to content assets and what business benefits they were realizing. Earley & Associates along with Taxonomy Strategies, launched a project to update this research. The goal of the study was to measure industry progress on a Metadata Maturity Model scale, a road map designed to help practitioners get started – and continuously improve – content tagging within organizations.

Preliminary results have shown that organizations that are more mature in metadata and taxonomy best practices outperform less mature organizations, with more mature organizations reporting findability and content management problems 10-15% less often. In addition to presenting results around maturity levels, this call will benchmark current search, taxonomy, and metadata practices against the 2005 survey and report on some surprising key findings around information access and best practices adoption.

Register here.

Free.  Register here

This session will cover issues in implementing an effective search experience and the range of available add-ons and compatible tools for extending SharePoint search functionality.

As the volume of information creeps steadily up in your SharePoint portal, so does the importance of sound navigation and metadata that can be leveraged for search. There are challenges around implementing hierarchical metadata and faceted search in SharePoint.

Lars Farstrup will discuss various options for creating an effective navigation and faceted search experience in Sharepoint. Paul Wlodarczyk will cover the range of available SharePoint add-ons and compatible tools to help you manage metadata, taxonomies and provide more robust search and tagging.

Topics include:

• Navigation pain points in SharePoint search
• Using faceted search for better navigation
• Build vs. buy scenarios for custom taxonomy & faceted search
• Taxonomy/Metadata/Search vendor landscape overview
• Specific tool features and limitations

If you’re going to be at the 2009 Semantic Technology Conference in San Jose next week, please stop by on Wednesday afternoon and listen to Richard and me present on integrating taxonomy and folksonomy.  This presentation by Earley & Associates was developed by Stephanie Lemiuex over the last several years, who was originally scheduled to present it.  She has developed four categories describing how taxonomy and folksonomy can be used together, and has collected a wealth of illustrative examples.   Richard is an excellent presenter (he has a Ph.D. in Ontology!); I’m honored to share the dais with him and to have the opportunity to present Stephanie’s fantastic content.  

Here’s the official abstract:

Tagging isn’t new – it’s been around for a dog’s age in internet years.  But in the past few years some fresh ideas and tools have reinvigorated the social tagging world.  These new approaches include an attempt to improve findability through a bit of structure and control.  While the idea of adding control to folksonomy seems like going against the whole selling point of social tagging (flexibility, openness), it is bringing the tagging to a new level, making it more viable for practical use in enterprises.  This session will present hybrid approaches to formal taxonomies and social tagging.  How can they be used in the corporate environment?  What type of content is appropriate for social tagging?  What kind of software is available for the enterprise?  Learn how social tagging is not necessarily anathema to corporate taxonomy programs and how this hybrid approach can bring the best of both worlds: a fresh, up to date taxonomy with the structure needed to improve information findability.
Key Takeaways:

• Folksonomy and taxonomy defined
• Drawbacks of pure social tagging
• Social tagging in the enterprise
• Hybrid taxonomy & folksonomy approaches: Four models

Take the Survey

Depending upon context, “tagging” can mean one of three different things: tagging a document, tagging within a document, or tagging a content object.

Tagging documents.  These days most of us think of tagging as the keywords we put on our documents – like our photos and websites – so that others can find them when they search.  User tags are fine for finding photos in flickr, but for tagging to be effective in business we need to make it systematic, so that we avoid ambiguity and improve search recall and relevance.  So we’re increasingly “mature” in our approaches to tagging: We use taxonomy to organize our terms into classes and to manage the relationships between terms.  We develop thesauri and foreign language equivalents.  We integrate taxonomies and thesauri into search indexes for ECM and site search and SEO.

Tagging within a document.  I got interested in tagging in the early days of XML (back when we spelled it “S-G-M-L”), when we were tagging within documents.  By tagging unstructured content inside documents we could do really sophisticated things – not just multi-channel output.  For example, knowing that a paragraph in a document was a step in a service procedure or that a string of gibberish was a part number let us bring life to that content when we transformed it from markup into an interactive electronic technical manual.  Tagging let us turn books into diagnostic software.

Tagging reusable content objects. As content reuse matured with standards like DITA, organizations had more reusable components, with more people creating them in more departments.  Tagging reusable content objects became essential to actually reusing them – if you couldn’t find it, you’d never reuse it.  If you had a single service manual with 100 procedures, now you have at least 100 reusable content objects, so the search scope increased by two orders of magnitude.  At IBM, colleagues report having over a million DITA topics in more than six repositories, with over a dozen departments sharing content across thousands of publications.  Searching for content objects is like trying to find a needle in a haystack, except you’re trying to find the right needle, and you have more and smaller needles to search amongst, in more and increasingly bigger haystacks.

Measuring Metadata Maturity.  Each type of tagging can have measurable benefits on your business.  Five years ago, Earley & Associates and Taxonomy Strategies developed a survey to understand metadata maturity for various types of businesses.  Earley is conducting an updated survey to see how organizations have moved up the learning curve.  Since we have a baseline of responses from five years ago, we’ll be able to describe how metadata and taxonomy practices have matured over time.  Also, the original survey was focused on the impact of metadata best practices on knowledge management and e-commerce search.  We now recognize that metadata is also used by technical communicators – especially those that use XML and other technologies to create, manage, and multichannel publish reusable content.  We want to hear from you all for the first time.

The survey is pretty detailed, so you might want to grab your favorite caffeinated beverage before you dig in.  As compensation for your time (about 15 minutes) Earley & Associates is offering these nifty incentives:

• A free pass to any future Earley & Associates Community of Practice conference call (a $50 value).  These are monthly, and the next one is Wednesday June 2^nd on Taxonomy for Portals featuring Giovanni Piazza, Chief Knowledge Officer of Ernst & Young, and Ralph Poole of Earley & Associates.
• A $200 discount on registration to the Henry Stewart conference on digital asset management, June 1-2 in NYC.  Seth Earley will be there presenting preliminary results.
• Free participation in a webcast reviewing the results of the survey (date TBA).

Take the Survey

Google Squared is Google’s answer (or perhaps one of their answers) to semantic search and Linked Data.  ‘Squared gets its moniker from the matrix used for displaying results – each “square” in the matrix contains some fact derived from the content on the source site. 
Each row in the Google Squared matrix is a search result, but the interesting part is the columns.  ‘Squared relies on RDFa and microformats on the source sites to extract structure for the search category – if it’s available (I’m not entirely sure how ‘Squared derives its semantic structure in the absence of metadata, but clearly it does).  So a search on “rollercoasters” will generate columns for height, speed, construction, etc.  Essentially, ‘Squared is generating search facets on the fly using structure that is implied or explicit in the set results set. 

Because sites are inconsistent with the amount of structure they provide, ‘Squared can – and will – make errors in interpreting free text.  For example, in the video, we can see that “height” – while intended to describe the height of the rollercoaster – sometimes returns text about the minimum height requirements for riders.  Still, from the demo ‘Squared does look interesting.  Predictable categories like Restaurants (e.g. a search on “pizza”) have the dimensions you’d expect to see in columns – description, address, price range, ambiance, etc. 

Google Squared affords general-purpose faceted search, because the columns can be used to refine the search results.  The current alpha doesn’t let you sort on a column, but clearly this is where things are heading.

Google Squared will be made publicly available on the Google Labs site in the next week or so. 

Here is the video courtesy of TechCrunch (over six minutes and shaky-cam – but you’ll get the idea).

As a footnote, TechCrunch declares that ‘Squared “is how Google will crush Wolfram|Alpha”.  I’m sorry – they missed on that point by a country mile.  Alpha isn’t a search engine – it’s a user experience (and a cool one at that) built atop a “curated” database, designed to answer queries that are primarily computational in nature.  Google searches the web.  No comparison – the products aren’t in the same class, and don’t solve the same problem.  Google can’t plot the Julia Set for you – Alpha can.  That doesn’t mean Alpha “crushes” ‘Squared, either.  Geez.   Anyway, more on Alpha later.