Molecular Content and the Separation of Concerns

Our current ways of writing and producing web content seem ill-prepared for the needs of the future.  Content producers focus on planning articles or web pages, but existing approaches aren’t sufficiently scalable or flexible.  Web publishers need to produce growing quantities of increasingly specific content.  High volume content still requires too much human effort: the tedious crafting of generic text, and/or complicated planning that often provides inadequate flexibility.    Content producers, using tools designed for creating articles, lack a viable strategy for creating content that machines can use in new contexts, such as voice interfaces.  Before audiences see the content, machines need to act on it.  It’s time to consider the machine as a key audience segment, instead of as a incidental party.

Within the content strategy community, a discussion is starting about making content “molecular”.  Content molecules are  fundamental building blocks that can be combined and transformed in various ways.  The concept still lacks a precise definition.  But it seems  a compelling metaphor for thinking about content, given the diverse directions content is being pulled.   Molecules connect together, like Tinkertoys™.  Forces (business, technical, and consumer preferences) are pulling content in different directions.  How to connect molecules of content together is a pressing issue.  The metaphor of molecular content offers a chance to reimagine how content is created, and how it can serve future needs.

“A molecule of content is the smallest stable autonomous part of content, with a unique purpose. Molecules, of varying purpose, can be built into stable compounds of content in order to form meaning and provide a purpose.” — Andy McDonald and Toni Byrd-Ressaire

Content needs to serve many functions.  It must provide coherent narratives as it always has done.  But it is also needed in short bursts as well.  Content will need be interactive: responding to user requests, anticipating needs, updating in real time as circumstances change.  Molecular content is fundamentally different from previous ideas of “modular” content.  Modular content are static standalone chunks of words.  Molecular content is data-aware: responsive, interactive and updatable.  Molecular content is connected to logic, and gets involved in the context it is used.

In the future we can expect content will need to be:

  • Able to be speak and listen
  • Capable of animation to show a change in state
  • Able to communicate with machine sensors
  • Capable of changing its tone of voice according to the user’s state of attention.

The notion of shape shifting words seems fantastic — especially to authors accustomed to controlling each word as it appears in a text.  But words are really just a special form of data — meaningful data.  Computers can manipulate data in all manner of ways.

Content for human communication can be more complex than common computer data.  Existing data practices will contribute to the foundations of molecular content.  But they will need to be extended and enhanced to support the unique needs of words and writing.

Writers don’t like considering words as data.  They raise two objections.  First, they consider words as more nuanced  than data.  Second, they worry that the process of writing will resemble programming.  Their concerns are valid.  No progress will happen if solutions are too rigid, or too complex.  At the same time, writers need to prepare for the possibility that the process of web writing will fundamentally change.

Four Hidden Activities in Writing

Writing is a tacit process, rarely subject to analytic scrutiny.    We’re aware we form sentences involving subject, verbs and objects.  These get joined together into paragraphs, and into articles.  But the process can be so iterative that we don’t notice separate steps. When writers talk about process, they generally refer to rituals rather than workflows.

If we break down the writing process, we see different activities:

  1. Making statements (typically sentences)
  2. Choosing the subjects to write about in statements.
  3. Organizing these statements into a flow.
  4. Making judgments through implicit references.

Implicit references are the stealthiest.  In our speech and writing, humans summarize thoughts.  We may make implicit statements, or render an explicit judgment that saves us from having to list everything (fro example, saying “the best…”, or “good…” ).  People who work with data talk about enumeration — basically, creating a definitive list of every value (e.g, the seven days of the week.)  When we talk, we assume shared knowledge rather than repeat it.  We assume others don’t want a complete list of every city in a country, they just want to know the largest cities.

If anything, technology has made the writing practice blur together these activities even more.  When we can rewrite on the screen, we can cut and replace with abandon.  The stringing together of words becomes unconscious.  Any attempt to make the process more explicit, and managed, can feel limiting, and is often met with resistance.

One of the big unanswered questions about molecular content is how to write it.  Molecular content will likely require a new way of creating content. First, we need to examine the process of web writing.

The Three Writing Workflows

Any web writing process needs to address several questions:

  • What things (proper nouns or entities) do you want to discuss?
  • What statements do you want to make about these things?
  • How to structure these statements (what template to use)?

These questions can be addressed in three different sequences or workflows:

  1. Author-driven
  2. Template-driven
  3. Domain-driven.

The author-driven approach treats writing as a craft, rather than as a process. The sequence starts with a blank page. The author writes a series of statements.  He structures these statements.  Finally, he may later tag things mentioned in the text to identify entities.

The author-driven sequence is:

  1. Write statements
  2. Structure text
  3. Tag text

The template-driven sequence starts with a template.  This approach is gaining popularity, with products like GatherContent providing templated forms that authors can fill-in.  The structure is pre-determined.  It is not unlike filling in an online job application.  The author needs to add text inside boxes on a form.  Later, the text can be tagged to identify entities mentioned.

The template-driven sequence is:

  1. Choose template or structure
  2. Write statements
  3. Tag Text.

The template-driven approach can sometimes allow the reuse of some blocks of text.  But often, the goal of templates is to organize content and facilitate the inputting of text.  A common organization can provide consistency for  audiences viewing different content.  But such structure doesn’t itself reduce the amount of writing required if the content has a single-use.

The domain-driven sequence starts by choosing what entities (people, places or things) the author plans to discuss.  Entities are the key variables in content.  They are what people searching for content are most likely to be seeking. Shouldn’t we know what entities we will be talking about before we start writing about them?  Once the entities are chosen, authors write statements about them.  They consider what can be said about them.  Writers can organize these statements by associating them with containers that provide structure.  Unlike the other approaches, authors don’t need to worry about tagging, because the entities are already tagged.

The domain-driven sequence is:

  1. Choose entities (pre-tagged)
  2. Write statements
  3. Choose containers for structure.

Domain-driven content writes content around entities. In other approaches, identifying what entity is mentioned is an afterthought, during tagging.  Because it is entity focused, domain-driven content is well matched to the needs molecular content.

Molecular Content through Domain-driven Writing

Domain-driven content is not new, but it is still not widely known.  I wrote about content and domains as they relate to Italian wine several years ago when I was living in Italy.  Happily, a new book has just come out by Carrie Hane and Mike  Atherton called Designing Connected Content talks about domain-driven content in detail.   It’s an excellent place to start to learn how to think about planning content from a domain perspective.  For writers wanting to understand how domains can influence writing, I recommend the blog by Teodora Petkova, who has been writing about this topic extensively.

Domain-driven writing may seem hard to envision.  How will one choose entities to write about before writing?  Perhaps writers could tap to select available entities, much like they tap to select available airline seats.  The tool could be connected to an open source knowledge graph that describes entities; a growing number of knowledge graphs are available.

The entities selected could be at the top of a screen, reminding the writer about what they should be writing about.  It could help writers remember to include details, or explore connections between topics they might not think to explore.  A tool could even helpfully offer a list of synonyms for entities, so that writers know what vocabulary they can draw on to discuss a subject.  Maybe it could even recommend some existing generic text about related entities, and the author could decide if that text is appropriate for discussing the entities they have chosen.

Domain-driven writing is a good fit for factually rich content.  It’s not an approach to use to write the next great novel.  Novelists will stick with the author-driven, blank page approach.

Much web writing is repetitive.  The details change, but the body of the text is the same. Domain-driven content puts the focus on those details.  It isolates the variables that change, from those that are constant.  In bring attention to the context that variables appear.

Even when the text of statements changes, domain-driven content allows factual data to be reused in different content.  If a product is mentioned in various content, the price can always appear beside the product name, no matter where the product name appears.

Two approaches to domain-driven writing are available, which can be used in combination.  The first approach creates reusable statements that are applicable to many different entities. The second approach allows custom statements for specific entities.

The first approach aims to standardize recurring patterns of writing by:

  1. Decomposing existing writing into common segments or chunks
  2. Normalizing or standardizing the text of these segments
  3. Reusing text of segments.

Instead of writing to make the text original-sounding, writers focus on how to make the text simpler by reducing the variation of expression.  The emphasis is on comprehension.  It is not about boosting attention by aiming for originality or the unexpected.

Let’s imagine you have a website about caring for your dog.  You have content about different topics, such as grooming your dog, or training your dog.  Dogs come in different breeds.  Does the breed of dog change anything you say about training advice?  Do you need to customize a paragraph about training requirements for specific breeds?  The website might have a mix of generic content that applies to all breeds, with some custom content relating to a specific breed.  The delta in the content helps to flush out when a specific breed is a special case in some way.

The second approach is start with entities — the key details your content addresses.  Instead of thinking about grand themes and then the details, you can reverse the sequence.  In our dog website example, we start with an collection of entities related to dogs.  This is the dog domain.  What might you want to say about the dog domain?  A tool could help you explore different angles.  You might choose a breed of dog to start with, perhaps a poodle.  The tool could show you all kinds of concepts connected to a poodle.  These concepts might be start of statements you’d want to make.  The tool would resemble a super-helpful thesaurus.  It would highlight different connections.  You could see other breeds of dog that are either similar or dissimilar.  Seeing those entities might prompt you to write some statements comparing different breeds of dog.  You might see concepts connected with dogs, such as traveling with dogs.  You could even drill down into sub-concepts, such as air travel or car travel.  The experience of traveling with a dog by air is different according to breed: for example, a Dachshund verses a Saint Bernard.  If you need to write statements to support specific tasks, the domain can help you identify related people, organizations, things, events, and locations — all the entities that are involved in the domain.

Domain-driven content is scalable.  You can start with statements about specific things, and then consider how you can generalize these statements.

Slide from a presentation by Rob Gillespie

Molecular Content: How Content Gets Liberated

Molecular content needs to be highly flexible.  To deliver such flexibility, different components need to play well with the rest of the world. One can’t overstate the diversity that exists currently in the web world.  Millions of individuals and organizations are trying to do various things, developing new solutions.   Diversity is increasing.  New platforms, new syntaxes, new channels, new programming languages, new architectures must all be accommodated.

We need to let go of the hope that one tool can do everything.  Tightly-coupled systems are seductive because they seem to offer everything you need in one place. Tightly-coupled systems give rise to authoring-development hybrid tools such as Dreamweaver or FontoXML.  Content, structure and logic all live together in a single source, which seems convenient.  But your flexibility will be limited by what those tools allow you to do.

The ideal of single sourcing of content is becoming less and less viable.  Requirements are becoming too elaborate and varied to expect a monolithic collection of files following a unified architecture to address all needs.  A single model for publishing web content can’t cope with everything being thrown at it.  Models are brittle.  We need systems where different functions are handled in different ways, depending on shifting circumstances and diverse preferences.   When you use a single model, others will reject what’s good about an approach because they hate what’s limiting about it.

Web publishing is becoming more decoupled.  Headless CMSs separate the authoring environment from content management and delivery.  Content management systems offer APIs that allow unbundled delivery of content.  Even the authoring process is getting unbundled, with new tools that specialize in distributed input, collaborative editing and offline workflows.

While the trend toward decoupling is gaining momentum, most attempts are limited in scope.  The don’t fundamentally change how content is created, or how it can be available.  They rely on the current writing paradigm, which is still document focused.  No one yet has developed solutions that make content truly molecular.

Molecular content will require a radical decoupling of systems that process content.  The only way to create content that is genuinely future-ready is to remove dependencies that require others to adopt legacy approaches and conventions.  Systems need to be adaptable, where parties involved in producing web content can swap out different sub-processes as new needs and better approaches emerge.

The Backend of Molecular Content: Separation of Concerns

It is challenging to talk about a concept as novel as molecular content without addressing how it would work.  I want to introduce a concept followed by developers called “the separation of concerns” and discuss how it is relevant to content.

Suppose a developer wanted to code a heading that said “Everyone is talking about John.”  In old-school HTML, developers would hard-code content, structure, and in-line logic together in a single HTML file.  Here’s what single source content might look like:

<html>

<body>

<h1>Everyone is talking about<div id="person"></div>.</h1>

<script>

document.getElementById("person").innerHTML = "John";

</script>

</body>

</html>

The file is hard to read, because everything is smushed together.  In a single file, we have content, structure, a variable, and a script.  It may sound efficient to have all that description in one place, until you realize you can’t reuse any of these elements.  It is brittle.

In modern practice, webpages are built from different elements: content files, templates, and separate scripts providing common logic.  Even metadata can be injected into a webpage from an outside file.  This decoupling allows many-to-many relationships.  One webpage may call many scripts, and one script may serve many webpages.  This is an example of the separation of concerns.

To separate concerns means that code is organized according to its purpose.  It is easier to maintain and reuse code when common things with similar roles are grouped together.

Let’s consider the different concerns or dimensions of how content is assembled.  The dimensions that computer systems consider to assemble content is in many respects similar to how authors assemble content.  They are:

  • Content variables
  • Narrative statements
  • Containers for content
  • Logic relating to content

These different concerns can be managed separately.

Variables

Variables are the energy in the content.  Because they vary, they are interesting.  Humans are hardwired to notice stuff that changes.

Variables live within statements.  Suppose wish aloud to our companion, Google, on our smartphone.  We say: “Ok, Google.  Get me a flight between Paris and Hong Kong for less than $500 in the first week of March.”  We have numerous variables in that one statement.  We have destinations (Paris and Hong Kong), price ($500) and time (first week of March).  Which of those is negotiable?  When we think about variables as being subject to negotiation, we can see how statements might change.

Variables animate statements the way atoms animate molecules, to use a metaphor.

Variables are frequently proper nouns or entities, which are visible in the content.   Such variables are descriptive metadata about the content.  A price mentioned in a statement is an example.

Some variables are not visible.  They are background information that won’t show up in a statement, but will be used to choose statements.  Such variables are often administrative metadata about content.  For example, to know if a statement is new or old, we could access a “published on” date variable.

What makes variables powerful is that they can be associated with each other.  These are not random words like the ones  used by a random phrase generator.  Variables follow patterns, and form associations.

For example, if we wanted to describe a person, we start by thinking about the variables associated with a person.

Person :

  Name: John,

  Gender: Male,

  Profession: Painter

A different person will have same variables, but with different variable values.  We can keep adding variables that might be useful.  This is the factual raw material that can be used in our content.

An important point about variables is that they can be represented different ways.  Because we want to separate concerns as much as possible, the variable lives separately from statements, instead of being embedded in them.  Because variables are separate, they can be transformed to serve different needs.  We don’t worry about what syntax is used.  It could be JSON, or YAML or Turtle.  When variables exist separately, their syntax can be easily converted.  We also don’t worry about what schema is used.  We can use different schemas, and note the equivalences between how different schemas refer to a variable.  We can reassign the name of a variable if required.  Maybe we want to refer to a person’s job instead of a person’s profession.  Not a problem.

Statements

Statements are generally text, though they could be an audio or video clip, or even an SVG graphic.  I’ll stick to text, since it is most familiar and easiest to discuss.

Statements will often be complete sentences, perhaps several related sentences.  But they could be shorter phrases such as a slogan or the line of a song.  Statements can be added to other statements.  Each line of a song can be joined together to produce a statement conveying the song’s full lyrics.

Statements become powerful when used in multiple places. Statements can accommodate visible variables to produce statement variations.  Some statements won’t use variables, and will be the same wherever they appear.

Statements are the basic molecules of content.  Some statements will be short, and some will be long.    The length depends on how consistent the information is.  We can use variables to produce statement variations, but the statements themselves stay consistent.  When we need need to talk about certain variables only in some situations, then new statements are needed.

Let’s look at how statements can incorporate variables.  We will use the person variables from our previous example.

Statement_1 : “Everyone is talking about {Person.Name}, the popular {Person.Profession}.”

Statement_2: “{Person.Name}’s Big Moment”

These are two alternatively worded statements that could be made about the same person.  Maybe we want to use them in different contexts.  Or we want to test which is more popular.  Or maybe they will be both used in the same article.  Because these statements are independent, they can be used in many ways.

I’ve used “pseudocode” to show how variables work within statements.  If we have many persons, we can be selective about which ones get mentioned.

But the syntax used to represent the text can follow any convention.  It could be plain text, or a subset of Markdown.  We are only interested in representing the information, not how it is structured or presented.  The information is independent of structure.  There’s no in-line markup.

Structure

Structure is how statements are arranged and represented.  Structure is one of the two ways that content molecules get “bonded” to other molecules (the other way is logic, to be discussed next).

Statements and structures have a many-to-many relationship.  That means the same statement can be used in many different structures, and a single structure can accommodate more than one statement.

A simple example (again using pseudocode) will show how statements get bonded into structures.  It is as simple as dropping the statement into a structural element.

/// Structure_1

<h1> {Statement_1}</h1>

/// But it could be instead

/// Structure_2

<h2> {Statement_1} </h2>

A single statement could be applied to many structures, including image captions or email headers.

As we consider a wider range of content, we can see how statements need to be used in different templates.  For example, the same transcript may need to appear as text of interview, and as subtitles of a video.

Structure should not be hard-coded into statements the way XML markup and CSS-selectors tend to do.  That limits the reuse of statements.

Molecular content should be independent of any specific structure, and able to adapt to various structures. We need structure flexibility.  Statements need to change structural roles.  We are accustomed to thinking about a statement having a fixed structural role.

Logic

Logic provides instructions about what content to get.  It may be a script (a few steps to do), a query (a command to find values of a certain kind), or a function (a reusable set of instructions).

Logic processes content to characterize it.  For example, if the content is about the “top” movies this week, the logic does a query to determine and display what the top-grossing films are.  Logic allows computers to make implicit statements, just like writers do, which makes the text sound more natural.

“With content molecules, content is separated not only from the presentation, but from the business logic, that is from the way the content is processed and manipulated.”  Alex Mayscheff

Logic is another way content molecules can bond together.  When logic is applied to statements, logic plays a matchmaking role.

Logic can also be applied to variables.  It can help to decide the right values to include in a statement.

A common example is when a query of a database generates a list.  The query asks the top 10 best selling literary fiction titles, and a statement is returned with 10 titles in a list.

Logic can provide more than simply reporting data.  As software gets smarter, it will be able to make more natural sounding statements.

Consider a simple example.  If we know the gender of a person, we can create new variables indicating the appropriate person pronoun and possessive pronoun to use.  Expressed in pseudocode, it might work like this:

Function(genderPronoun)

If Person.Gender == Male

  Assign

    PersonalPronoun -> He

    PossessivePronoun -> His

Else

  Assign

    PersonalPronoun -> She

    PossessivePronoun -> Her

Endif

Logic can summarize variables so they are easier for humans to comprehend.  If we only rely on variables, we have to see the values exactly has they are recorded.   In earlier examples, the variable was directly injected into a statement.  The variable says: When you get here, put a certain value here.

Using logic, a variable can call a function.  The function instructs: When you get here, figure out the appropriate value to put here.  This gives much more flexibility for the scope of values that can be used in statements.

Because the logic is separated from the variables and the statements, we don’t care what form of logic is used.  It might be PHP, Python or Javascript.  Or a query language such as SQL or Sparql.  Or some new AI algorithm.  Developers might combine different programming languages, so that different ones can perform specialized roles.  It is a very different situation than exists when content is encoded in XML, forcing developers to rely on XSLT or some other XML-focused language.

Systematizing What’s Routine

My excursion into the coding of molecular content may give a false impression that writers will need to code in the future.  I hope that is not the case.  Nearly everyone I know agrees that code is distracting when appearing in writing.  Ideally, the separation of concerns means that code won’t appear in statements.

What’s been missing are systems that make it easy for writers to reuse facts (variables), statements (content chunks), and templates (structures).  Systems should let writers add some logic to their writing without worrying about the programming behind it, perhaps by choosing some pre-made “recipes” that can be dragged into text and inserted.  I’ve seen enough different efforts to simplify systems (from Jekyll to IFTTT to automated suggestions) to believe writer-friendly tools to support molecular content are possible.  But new systems emerge only when a large community believes there is a better way.  No one person, or company, can build and sell a new system, much less force its adoption.

When I started out working in the web world, all user interface screens were individually designed.  Each one needed to be crafted and tested individually.  Each screen was a precious creation.  Eventually, the UX community realized that approach was madness.  UX folks weren’t able to keep up with the volume of screens that users needed to see.  And UX staff were recreating the same kinds of screens again and again.  Eventually, the UX community adopted components, patterns and templates.  They created systems that could scale.  Original, new designs are needed only in highly novel situations, such as new device platforms or enabling interaction technology.  The rest can be reused and repeated.

Atom Design methodology by Brad Frost

UX designers now talk about a concept called atomic design.  Atomic design sounds related to molecular content.

The transformation of UX design is still ongoing, but it’s impressive in what has been achieved already.  One might expect designers would be resistant to technology.  Many studied graphic design in art schools, using colored markers.  When applying their graphics knowledge to web design, they saw the benefits of reusable CSS, adopted plug-and-play Javascript frameworks, and started building component libraries. Much of the progress was the work of various individuals trying to solve common problems.  Only recently have companies started marketing complete solutions for UI component management.  Designers still like to sketch, but they don’t expect screen design to be a manual craft.

I’ve long been puzzled why so many art school grads can happily embrace technology, while so many writers have an anti-technology attitude.  Designers have found how technology can extend their productivity immensely. I hope writers will discover the same.  A craft approach to writing is wonderful for novels, but insane for producing corporate web content.

Most of the original structured writing approaches built in XML are tightly-coupled, resulting in systems that are both inflexible and overly complex.  A more loosely-coupled system, based on a separation of concerns, promises to be more flexible, and can be less complex, since adopters can choose the capabilities they need and are willing to learn.  Designers have benefitted from open systems, such as CSS patterns, Javascript frameworks, and other publicly available, reusable components. Designers can choose what technology they want to use, often having more than one option. Writers need open systems to support their work as well.

—  Michael Andrews

How Content Can Answer Unanticipated Questions

How can publishers answer questions that audiences may have, when they don’t always know what will interest people? This is not a trick question. To be agile, publishers need to plan for flexibility.   They need to prepare content for scenarios they can’t anticipate in advance.

Content design has never been more important.  People have less time than ever to deal with unwanted content.  But content design should not be about spoon-feeding audiences answers to pre-approved questions.  Content design should instead empower audiences to consume the precise content they need.  Publishers should enable audiences to decide the answer that matches their need.  Publishers shouldn’t believe they can always anticipate what audiences need.  They can’t  always package content to match a  known need.  Recent developments in search technology are shaking up thinking about how to provide answers to audiences.

The Limitations of Questions as Templates for Content Development

Current practices presume a certain process.  We should start with a list of questions that users have, then write content answering those questions. The question will tell us what content to create. This approach, however, has limitations which may not be obvious.

I’ve long been an advocate and practitioner of user research.  It makes no sense to create content users indicate they have absolutely no interest in.  But user research is merely a starting point for considering user questions.  It should not be the final arbiter of what could be important to users.

“People are really fascinating and interesting … and weird! It’s really hard to guess their behaviors accurately. ” — Peter Koechley, Upworthy

Many user questions can’t be guessed — or discovered — in advance.  When doing user research, organizations can be over-confident about what questions they think users will have in the future.  User research probes the motivational level of interests and needs, rather than the more granular informational level of specific questions.  User research helps to  understand users, but it will simplify user needs into personas.  The diversity, and contextual complexity, that spawn the range of real word user questions gets smoothed over.  Qualitative user research data is too broad to uncover the full range of potential questions in detail.  Quantitative data analysis of past online queries can provide more granular insights, But even quantitative data won’t predict all situations, especially when novel situations arise.

Two common approaches to question-templated content development are:

  • The “top tasks” approach
  • The long-tail approach.

Some content strategists favor the top task approach  — especially those who focus on task-oriented transactional content.

Many SEOs favor the long tail approach — especially those who want to promote awareness-orientated marketing content.

The top tasks approach makes assumptions about essential user questions, based on past user behavior with a website.  An organization may decide that the top 10 search queries drive 90% of web traffic, so those 10 questions are the ones to offer answers.  Each question gets one answer.  It’s a rearview approach that assumes no curiosity on the part of audiences.  Audience needs exist only as an extension of their interaction with the organization.  All questions considered relevant relate to user tasks linked to that specific organization.

The hidden assumptions of the top tasks approach are:

  • Everyone has the same questions
  • Because everyone has the same questions, everyone should get the same answers
  • If different people start to ask different questions, publishers can ignore those questions, because they aren’t top questions.

Providing homogenized answers to homogenized questions is appealing to homogenized organizations.  Especially to  government offices, banks, or tech support units.  But cookie cutter content can seem like it’s created by a faceless organization.  Standardized answers don’t satisfy customer’s growing expectations. They expect more personalized service.

The long tail approach tries to anticipate user questions by crafting answers for many question variations.  Each variation addresses an ever narrower ranges of questions. The idea is to get an inventory of questions all kinds of people are asking, and then develop answers to all these questions, so there is something for everyone.  On the surface, this approach seems to deliver more individualized answers.  But we will see, that is not always the case.

Both the top tasks, and long tail, approaches assume that each question has one answer.  A content item exists to answer that one specific question.

In practice, the formula that one question has one answer doesn’t hold.   Different questions lead to the same content.  Type question variations on Google, and Google rewards you with the same links going to the same content.  Not all question variations are substantially different.  If you type “How to fly a kite” in Google, you can see related questions such as “How to fly a kite step-by-step” or “How to fly a kite by yourself”.  You’ll also find “long tail” questions such as “How to fly a kite with little wind” or even more optimistically, “How to fly a kite with no wind”.

The notion of a related search is vague.  It could be a search query that is essentially equivalent to another, but phrased differently.  It could be question that implies distinctions or details that may not be present in the information or that may not even be crucial.  Suppose we imagine content addressing “How to fly a kite for firefighters” and another on “Easy steps to kite flying for bus drivers”.  We’d likely find the essence of this long tail content is no different from the more general answer.  The idea that long tail content is necessarily more relevant is fiction.

The other characteristic of question-templated content is that the questions and answers are pre-assembled and frozen.  If we phrase a question differently, such as “What’s different about kite flying for bus drivers?”, we aren’t likely to get an answer.  At most, we’ll get content talking about kite flying that for some reason mentions bus drivers.  The content creator decides what content the reader will get, instead of the reader deciding.

Content design should be built on a foundation of compositional content.  What content is assembled and delivered can be based on the specific question asked.  Suppose you want to ask “How to tell someone to ‘go fly a kite’ ”?  When decomposed, the question reveals two distinct sub-questions.  One sub-question concerns how to deliver a message in general, covering tone or medium.  The other sub-question concerns what message alternatives are available about a specific issue — in this example, the desire to get someone else to change their behavior.

In principle, machines can assemble an answer to such a complex question, even though no person has created an answer to that specific question already.   The machine would draw on two components.  One would component address points to make about an issue; and the other component would address ways to deliver those points.

A compositional topic could be rich in variations that would yield different answers.  It could address: “How to tell a colleague…” or “How to tell a nosy relative…,” or whomever.  The answer could include components about the general aspects of the issue, which could be supplemented with some advice specific to the question variation.

For those familiar with structured content, the use of components to create content variations will seem familiar.  The difference here is that users initiate the assembly of components in novel configurations.  We don’t know in advance what the user wants, so we therefore have to provide them with the raw material to supply the answer to their unknown query.

Information Generates Questions

Part of the reason people can be unpredictable in their questions is that their interests and understanding evolve over time.  Sometimes the facts of a situation can change as well.

Laura E. Davis, digital news director of USC’s Annenberg Media Center, recently wrote about “Writing answers before you know the question.”   Her question flips the assumption that most writers have: that writers know reader questions ahead of time, and the task of the writer is to provide answers to them.  Most writers expect that information presented will follow the questions audiences ask.  But the reverse is also true. Information, or the expectation of information, sparks questions.  Sometimes writers will never have thought of the questions their readers might have.

Davis cites several trends that are making audience questions less predictable.  Audiences are becoming more conversational in how they access content.  Questions can unfold in a conversation, without knowing where they may lead.  Events can unfold quickly, and not conform to a tidy summary answer. These issues gain importance as conversational interfaces become more common.  “As we move forward, more and more, we’ll be writing answers before we know the question.”

In conversation, questions and answers flow spontaneously.  How can content become more spontaneous?  How can content prepare for a “zero UI” future, as Davis puts it?  We’ll look at two approaches, metadata and machine reading, which publishers can combine to offer laser precision in answers.

‘Literate machines’ will provide dynamic answers

Historically, questions asked online were answered by a list of hyperlinks.  Even today, many chatbots provide an answer by pointing to a hyperlink of content the reader must read.   When a computer points a user to a document title (in the form of a hyperlink), it generally is pointing the user to pre-assembled content.  Pre-assembled content runs a high risk of not being exactly what the user is looking for.

Yet the more recent trend is to provide answers directly, instead of answering queries by providing links to documents.  Everyone is familiar with Google’s instant answers. This approach is being adopted most of the other major tech companies as well.  How answers are being delivered is transforming quickly.

Advances in semantic technology and AI are allowing both questions and answers to become more iterative, and fluid.  Users may not consider a single answer to a question they pose as complete. They may want several pieces of information to give them a complete understanding.  To give users complete answers, machines stitch together several fragments from different source.  Audiences can ask clarifying or follow up questions to fill out their knowledge, and contextual answers will appear.

Semantic metadata facilitates machine discovery and understanding of information.  Metadata is powerful because it can relate information from different sources. Publishers can include their information as part of a relevant answer to a user query.  For example, suppose a user asks “What local cinemas are showing films made before 1960 this evening?”  There may not be a single item of content providing that answer.  But metadata from different content can assemble an answer.  The listings of local cinemas can be combined with data about films from a film encyclopedia (to filter by year).  The ability of metadata to assemble information from many sources upends the expectation of some publishers, who believe they must provide comprehensive information about topics to answer any audience question.  Instead, their goal should be to focus on providing comprehensive information that they are uniquely positioned to offer, and to link through metadata to other sources that provide related information that might arise in a question asked by users.

The question in this example may seem arbitrary — and it is.  Why would someone want to watch films made before 1960?  What special about 1960?  Why not 1965?  Or 1950?  Because the question, seen from the outside, seems arbitrary, no one will create content specifically to answer this question.  The variations in how the question could be framed are limitless.  Which is why metadata is powerful in providing answers to questions that may be infrequently asked, or have never been asked previously.  Just because a question is novel does not mean it is unimportant.

Given the quantity of content that’s created, someone may have written content that provides part of an answer to a question.  But that answer could be buried within a larger discussion that isn’t the focus of the user’s question.  If you are curious where a new film start grew up, there might not be specific content answering that question.  But he or she may have mentioned it in passing during an interview about their latest film.  How might you locate that information without reading various interviews in full?

Machine reading comprehension (MRC) is an emerging technique that promises to transform how content is used.  Its premise is simple but awe inspiring.  Machines can read texts just like humans do, and understand what the text means.  They can do this at incredible speeds, so that can locate specific statements quickly, interpreting what the statement means, relating it to questions or statement made elsewhere.   Machine reading does not require structure, but it presumably benefits from having structure.

Amy Webb at NYU demonstrated how machine reading comprehension works in a recent presentation (here at minute 34) . Reading a book, MRC can extract the meaning.  Yes, someday soon computers will be able to speed-read War and Peace and be able to tell us what the novel is about (beyond the obvious, that it’s about Russia.)

slide with text
Slide from Amy Webb presentation on machine reading comprehension (MRC) at ONA17 conference.

MRC has been a keen research focus of many firms developing audio interfaces.  Audioburst is a new service that digests the transcripts of audio interviews.  Users can ask Alexa a question about a news topic, and Alexa can query Audioburst to find snippets of content relevant to the query, and will combine and play back different audio clips from different radio programs related to the question.

Microsoft has been at the forefront of MRC research.   I want to highlight some of their work because they are combining MRC with semantic metadata in products that are widely used.

“We’re trying to develop what we call a literate machine: A machine that can read text, understand text and then learn how to communicate, whether it’s written or orally.” — Kaheer Suleman of Microsoft

Microsoft notes: “Machine reading comprehension systems also could help people more easily find the information they need in car manuals or dense tax code documents.”

MRC is being used in Microsoft products such as Cortana (the voice assistant similar to Alexa or Siri), and Bing (the search engine that competes with Google).

A recent news article states: “Microsoft’s virtual assistant Cortana will get an upgrade as well, allowing it to make use of machine reading comprehension to summarize search results. ”

Earlier this month, Bing announced it would use MRC: “Bing’s comparison answers understand entities, their aspects, and using machine reading comprehension, reads the web to save you time combing through numerous dense documents.”

screenshot of Bing blog post on MRC
How Bing uses machine reading to provide multifaceted answers based on text from different sources

 

For Bing users this means:

  • “If there are different authoritative perspectives on a topic, such as benefits vs drawbacks, Bing will aggregate the two viewpoints from reputable sources”
  • “If there are multiple ways to answer a question, you’ll get a carousel of intelligent answers.”
  • “If you need help figuring out the right question to ask, Bing will help you with clarifying questions.”

As the Microsoft examples highlight, the notion that there is only one best answer to a question is no longer a given.  People want different perspectives, and different levels of detail.  Literate machines can help people retrieve answers that match their interests.

Conclusion

Information-rationing is not in the best interests of content consumers.  Content strategists have long warned of the dangers of providing too much information.  But too much information isn’t necessarily the problem.  No one complains about Wikipedia having too much information.

My advice to content creators is this.  If you have unique information to share, you should publish it.  Even if you’re not sure whether users have a pre-existing need to look for that information, it could be valuable.  Self-censorship does not make sense.  At the same time, content creators should not feel they must create a complete or definitive presentation of a topic.  Increasingly, machines will be able to stitch together information from different sources for the benefit of users.  Content creators should focus on what they know best.  Duplicating information that exists elsewhere benefits no one.

We can’t predict what information people will need in the future. Content that is information-rich is worthwhile content.  We need to make such information accessible, so audience can retrieve it when it is be needed.  We need to help make machines literate.

— Michael Andrews