Category: Content Engineering

Content Structure in Tables

Post author By Michael Andrews
Post date November 28, 2016

Content used in tables requires planning. Some authors consider tables fussy and unnecessary, and assume readers find them confusing. Dislike of tables often reflects mistaken ideas about what content in a table is meant to convey. Many people treat tables as blank cells to fill-in as they please. Such free-form tables are the root source of many problems in large scale content publishing operations. Tables are most effective when their structure is designed to support the meaning of the content they display.

Contrary to popular perception, tables are not really a single content type. Various types of tables exist, each with distinct content structures. Unfortunately, the tools authors use to make tables, whether spreadsheets such as Excel or plain old HTML markup, encourage them to think of tables as a blank canvas on which anything can be added. Just choose the number of columns and rows you want, and a table results. Tables shouldn’t be considered merely as a display format. Tables should, where possible, convey the underlying structure of the content. Structure provides editorial guidance to readers so they can understand the content more clearly, and helps manage how the content within the table is delivered and reused in different contexts.

Over the past five to ten years, data scientists have focused research on how to extract the information embedded in millions of HTML tables published on the web. Researchers consider the data in HTML tables as “semi-structured”. These tables frequently follow predictable patterns, but are subject to great inconsistency as well. Even when the table follows a pattern, the structure is normally implicit rather than explicit.

Published tables of content should indicate an explicit structure that is clear to readers and to machines alike. To reach that goal, we need to understand the implicit structure of tables published on the web today. Many tables follow design patterns that reflect consistent content structures. These patterns can provide a the basis to design templates for tables that will be used consistently.

To perceive the implicit structure of content, think about the content as composed of three parts:

A subject or topic (which we will refer to as “S”)
A property or attribute of the subject (referred to as “P”)
An object or value of the property of the subject (“O”)

For example, we can identify the structure of the following statement: “Mary (S) knows (P) Jane (O).” The property announces some information about the subject that is revealed by the object of this statement.

Tables allow many statements to be expressed in a compact space. The S-P-O technique can be applied to tabular information. Sometimes the information in tables is more complex than the basic S-P-O structure, and some tables (such as a Sudoku puzzle) lack this structure entirely. Nonetheless, the S-P-O structure can help identify the underlying structure of content in many common types of tables.

Let’s consider the content structure commonly found in tables. No standard taxonomy of table formats seems to exist, so I will offer my own terms to refer to these structures. Five common kinds of tables are:

Mutual comparison tables
Dimensional tables
Alternative list tables
Spectrum tables
Matrix tables

These five kinds of tables are not the only kinds possible, and some authors or data experts will object that these examples limit options for arranging information. Yet it is important to simplify and standardize how information is displayed in tables when publishing at enterprise scale. Knowing widely used and effective patterns for tables provides a basis to develop standardized templates to display tabular information.

Mutual Comparison Tables

The mutual comparison is a very common table type. It lists a number of items (subjects) that all belong to a common category, and then indicates different properties and values for these items. Let’s look at some examples, starting with a table of most active stocks. Each company is a subject, and different properties of the company are identified in the column headings. The values (or objects) of these properties appear within each row. All the companies belong to a common category: most active stocks. It is common for the table heading of a mutual comparison table to refer in some way to the kind of subject listed, and one or more of the key properties associated with that subject. The table makes two kinds of statements. First, that the most active stocks include certain companies such as BAC and AMD. Next, a second kind of statement is made where a company, say BAC, has a price that has a dollar value. Each company in the table has multiple statements, presented in each column.

Let’s consider another mutual comparison table from the website FiveThirtyEight, showing sports teams rankings. While by convention the subjects in the table typically appear in the left column, in this table the subjects (the teams) appear in the third column. The properties of the teams appear on either side of the team’s name. The table heading only implicitly indicates what the different subjects in the table share in common: that they all belong to the NFL.

The archetypal content structure for a mutual comparison table is illustrated below. The arrows show the relationships between different elements in the table. The overarching subject, the category that the table discusses, is indicated by a S’. The individual subjects have one or more properties, each property generally having a single value (but not necessarily so). What the value means depends on the property that describes it, which in turn depends on the subject the property refers to.

Dimensional Tables

Dimensional tables are similar to mutual comparison tables, except that only one subject is discussed. They reveal dimensions of a single topic. Let’s look at a table from Wikipedia about winners of the Booker prize for literature. The subject of the table is the Booker prize.

The table has rows of information relating to the winner for each year. Although the table allows sorting by any column, the key column that defines how the content in a row is related is the year. We can say that the year property is the primary dimension, while other properties such as country (of the author) and genre are secondary dimensions. Such tables identify some primary dimension that varies as a way to structure the content. Typically the most important property of the subject will be the left column of the table. In many cases the primary property is one that is required to exist in order for other properties to also be present. The best way to think about dimensional tables is to consider that they involve a chain of statements. First, we announce that a Booker prize was awarded in 2016. If no Booker prize was given in 2016 (and some awards choose to skip years if they don’t like the candidates), then none of the other properties relating to a 2016 winner would make sense. Two kinds of statements are supported by the structure of the table:

A Booker Prize was awarded in 2016.
It was given to Paul Beatty.

Dimensional tables can be applied to qualitative as well as quantitive properties and values. Here’s an example that’s more conceptual. The subject of the table is strategic communication. Again we see that the columns are not of equal importance. The primary dimension relates to communication function, while other dimensions are structurally dependent on that. The structure of the table indicates that its author considered communication function as the key to understanding communication approaches, rather than say, channel.

The structure of a dimensional table is shown in the diagram below. The secondary properties and their values depend on the primary property.

Alternative List Tables

An alternative list table is similar to a dimensional table in that the table refers to one subject only. It differs in that each property addressed by the table is independent of the others. This means that the rows presenting values aren’t related. The following example of an alternative list table relates to spending decisions. Although the table has no title, the subject of the table is spending. The subject has two properties, which are alternatives. The table answers which kinds of purchases are major, and which are minor. Examples of each are listed under the columns. This is a common kind of table to display content, and is often used to compare products.

The following diagram shows the structure of the content in an alternative list table.

What can make alternative list tables difficult to interpret is that sometimes the creator of the table leaves out information, or implies relationships that may or may not be intended. Let’s look at another example, from the World Bank. The table discusses two alternative ways of thinking, automatic and deliberative. Examples are shown for each alternative. Are the examples just a random list, or do they suggest some additional dimensions?

Many tables using this format choose to leave out a column on the left that would explain what each value represents. In this table, a line is drawn across the values implying that each pair is related. For example, “narrow frame” and “wide frame,” or “effortful” and “effortless,” both seem related pairs. But what about “associative” and “based on reasoning”? Are those opposite or similar, and what exactly do these values refer to? What’s the difference between associative and intuitive, which are both properties of automatic thinking? When tables drop labels, the reader can’t understand the structure of the content presented in the table without consulting accompanying text. This prevents the table from being reusable in different contexts.

Spectrum Tables

A spectrum table is a special type that mixes elements from the alternative list (showing alternative kinds) and the dimensional (addressing distinct properties of a subject) models. When structured properly, it is a sophisticated way to present content.

A spectrum table answers the question: how does a value for a property vary according to some other factor? One set of properties are treated as dependent variables (values change depending on the property considered), while the other set are treated as independent variables. A concrete example will illustrate how the structure works. We can see the service hours provided increases with the price of the monthly package.

In the left column are all the features that could be part of a service offered for sale. The other columns represent different price options, and the table reveals how much of each feature is offered according to price.

The following diagram illustrates the structure of a spectrum table. The column heading will be of the same data type, which allows them to be compared directly.

Most of us are familiar with the pattern shown in the example. But not all product comparison tables are spectrum tables. Many are alternative list tables. Marketers often remove any labels explaining what the values refer to, which allows them to make apples to oranges comparisons, so that the values of every option often sounds positive, even through they many not be addressing the same properties. It’s a disingenious way to hide information when some options lack certain features. Such tables can never be constructed on the basis of features offered, since there are no rules governing what is shown. They are brittle and one-offs. It is not an approach that scales well.

Matrix Tables

Many tables appear to be matrices, because they contain both row and column headings. A genuine matrix table has unique structural characteristics. Its defining characteristic is that both row and column headings are of equal importance. Each value has two properties. Matrix tables are not common in web content, but can be useful for certain situations, such as classifying concepts. For example, e-learning content might use matrix tables.

First, let’s look at a quasi-matrix. This example from the Sloan Management Review on first appearance seems like a matrix, but actually isn’t. The table has a simple structure, showing percentage values, with both column and row headings of seeming equal importance. On closer inspection, however, the table is actually a mutual comparison table. Although not explicitly noted, the table emphasizes the percentages in the rows representing industry sectors, rather than the columns representing technologies. The percentages in the rows add up to 100%. The row of the table show the relative interest of each industry in different technologies, but the columns don’t show what the relative interest of a technology is among industries. If the survey’s answers were translated into investment forecasts, the table might suggest what percentage of new investment by the retail industry will go into analytics, but it won’t suggest what percentage of spending on analytics will be made by the retail industry. Hence the table is not truly bidirectional in structure.

A genuine matrix can start with a value, and answer two separate questions. Consider the following example, from the World Bank. It answers what kinds of relationships different social networks involve. The relationships involve two independent facets, neither of which is more important than the other. We can look at a friendship network such as Facebook and learn both the type and direction of the ties it involves (assuming of course we understand the terminology used in the table).

The structure of content in a matrix table is presented below. Matrix tables describe two facets of a subject, and explore alternative categories for each facet. Each value will have two properties, which together describe the subject. In terms of the example, we can see that a social network that has both explicit and directed ties is called a friendship network.

Standardizing Tables

Tables presenting content should be planned according to long-term audience and business needs. Unfortunately, ad-hoc tables are far too common. Problems arise when:

Audience-facing tables are not designed around their needs, but are simply generated on demand from queries.
Tables are hand-crafted without thought to their wider use.

For database experts who think about tables in terms of rows and columns, an endless variety of tables can be generated. Much data-centric content suffers from looking like a raw database. Having many variants can be useful for individuals needing highly specific reports, but an anything-is-possible approach lacks editorial oversight. Audiences need tables that explain and compare key variables influencing their decisions. They want to have confidence they both understand, and know they are not missing any key information.

The other kind of unplanned table is created by authors who design on their own to fit a specific need. The problem is especially common in content marketing. Idiosyncratic tables routinely drop explanatory headings, and sometimes add extraneous rows or columns that aren’t directly related to the subject of the table. Authors focus on how to emphasize the wording of content that appears in tables to attract attention to specific items of information. They design online tables as if they were PowerPoint slides, centered on specific messages, instead of representing the content as a whole. Their tables don’t consider how the content may need to be revised and reused in the future. Audiences can find glib tables confusing and untrustworthy.

Content structure is discovered by deconstructing examples. For those involved with content design and content engineering, the first step is to take an inventory of tables used in content published. Look for patterns in tabular content, and standardize these patterns in templates. Remove unnecessary variation, and designs that can’t be used widely. Standardized tables allow content to become more flexible. Templates based on standard table structures will make initial publication and subsequent reuse and updates easier.

— Michael Andrews

Tags tables

Content Engineering

Sorting to Prioritize the Content Experience

Post author By Michael Andrews
Post date October 24, 2016

The act of sorting seems so familiar you may think little about it. We sort our possessions to organize them. Some people even sort the sox in their drawer according to color or occasion. We sort to organize things, and more fundamentally, to prioritize our content. When considered in relation to IT, sorting is the programmatic prioritization of content using a simple ordering procedure. Simple sorting routines can offer much value, even if more sophisticated techniques to prioritize content are available.

Most discussion of sorting focuses on its technical dimensions — the rules for sorting correctly. Developers study various algorithms to optimize sorting. Editors must follow detailed rules to correctly sort entries appearing in indexes. Interaction designers focus on how to implement sorting options on user interfaces. Sorting is also utilized in statistical operations, though the strict criteria applied in statistical analysis is different from how people will think about sorting content.

In contrast to its technical dimensions, the experiential dimensions of sorting receive less attention. Besides considering how to sort items, we must also consider why audiences want items sorted in different scenarios. Every day, when we write lists, we make decisions that reflect our understanding of the purpose of sorting . Do we present the list as unordered bullets, or do we number the list items? If we number the items, what do the numbers represent? All of our content faces such existential dilemmas — indicating to others how items are prioritized.

Interaction designers often assume that users will want to sort items appearing in a list. Such an assumption confuses a want with a need. In many cases users don’t want to interact to get specific views of content. They may need different views, but don’t want unnecessary work to get those views. Consider the below screenshot, from a website devoted to user interface design patterns. A spreadsheet paradigm is imposed on web content. Audiences are given the option to sort content on any criteria, but have no guidance about which criteria are important.

A sorting UI pattern. Source: styleframework.com

This pattern is widely implemented. The user interface presents the illusion of control, but offers little to help audiences understand insights in the content. It presents unnecessary work for audiences, and provides an uncertain payoff. Screens like this continue getting made because sorting functionality is ubiquitous and easy to implement. Adding it seems cost-free. The publisher never did the hard work of asking why the audience wanted the information sorted. What value does a sort offer the audience? How can computers provide such sorting without requiring the audiences to specify it manually?

Sorting tends to be discussed as being a widget and labeling issue, debating the options that are possible, and the user confusion that can result from having those options. Instead of worrying about the clarity of the widgets and labels, a better approach is to make editorial decisions that remove that complexity, and focus on the value sorting can offer.

Sorting practices can be explored in terms of five goals they support:

Sorting to Locate
Sorting to Rank
Sorting to Sequence
Sorting to Sample
Sorting to Profile and Evaluate

While these approaches differ in emphasis, they share a common goal of prioritizing content by making a judgment about what’s important to the audience. Users prefer sorted content because it reduces the amount of content they need to view and consider. Sorting doesn’t remove content: it highlights certain content, allowing other content to be ignored. Sorting should make using content easier for audiences, which is why that task needs to be delegated to computers whenever possible.

Prioritizing by Index Value: Sorting to Locate

Indexes are markers used to locate content. With indexes, the chief role of sorting is “findability”.

Alphabetic sorting is the most widely used form of sorting. Though clearly useful, alphabetic sorting’s value is sometimes presumed when it has none. In many situations, alphabetic sorting merely provides a semblance of order without providing any true value beyond psychological comfort. The chief value of alphabetic sorting arises when the user already knows about an item and expects to find it on the list. It can help locate the item, and confirm its inclusion. Developers have use for reverse alphabetic sorting, but audiences rarely benefit from reverse alphabetic ordering. Except in rare cases, it makes no sense to offer audiences a choice of sorting either in ascending or descending alphabetical order.

Another kind of locational sorting involves sorting items into nominal (named) categories. For example, the fact checking website PoliFact identifies statements made by American politicians to see if they have “flipped” their previous position. One can sort statements according to whether is a statement is: 1. Partially (half) flipped; 2. Fully “flopped”; or 3. Not flipped. Such sorting helps audiences locate statements they might be interested in, to judge if a view is simply getting new attention, or whether it is a new position.

An item sorting according to category "Fully flopped". Source: PolitiFact — An item sorting according to category “Full Flop”. Source: PolitiFact

Prioritizing by Frequency: Sorting to Rank

Sorting is especially useful with quantitative values. Sorting can rank items on the basis of numeric values associated with the item. Sorting on the quantity determines the ordinal ranking.

In contrast to alphabetic sorting, descending order (from high to low) is frequently useful for quantitative values. People are often looking for content relating to the highest rated item, or best performing one.

In addition to sorting by explicit quantitive values such as price, less obvious applications of rank-based sorting exist that utilize hidden or implicit information. Behavioral values can summarize activity relating to the content, such as when articles are sorted by number of comments. The most common form of behavioral values are consumption-related popularity values, These come in many forms, and are widely used to sort content. Examples included sorting by:

Best selling
Most viewed/played
Most recommended
Most downloaded
Most frequently bought together

Another use of quantitative sorts is to create “buckets” that rank content in summary form. Buckets are derived data that are not explicitly visible in the content, where clustering (summarizing frequency) is performed in tandem with sorting. Publishers create buckets covering ranges of values to summarize how frequently items appear. Items are sorted into buckets defined by ranges (e.g., Number of items “below $100,” “$101 – 499,” “$500 and above”). When applied to the sorting of content, these intervals don’t need to be equal in size.

In addition to ranking by a single criterion, publishers can provide different ranking perspectives that consider alternative criteria. Alternative-criteria rankings can be complex, so care is needed to hide the complexity from audiences. Imagine that content addresses five products: A, B, C, D, and E. A consumer ratings website might sort-rank items according to different criteria:

Best picks for budget buyers: A, D
Best picks for power users: E , C
Best picks for novices: A, B

In some respects, this kind of sorting is similar to the sorting of columns in a spreadsheet. In a spreadsheet, one can rank alternative criteria by selecting different columns to reorder rows, to see how rankings change when various criteria are considered. What’s different here is that editorial choices are being made instead of forcing the reader to decide what’s important to focus on. Each category represents a theme rather than formal attribute. For example, what’s best for power users might be a combination of the number of features a product offers, and the extent the performance of key features are above average. That kind of score can be computed behind the scenes. Once items are ranked, only the top two items are presented, to keep the focus on what’s best in each category.

An alternate-criteria sort, where items are ranked according to best overall and best value. Source: Consumer Reports, via Minonline.com

Another ranking pattern is the “Top N by category” pattern. Many publishers will curate content according to how the content ranks within different categories. A news site might present a list of top five articles in sports, in health, in politics and in business. The curatorial decisions relate to what categories are most important in the larger body of content, how many items (N) to show, and on what basis (number of views, comments, etc.)

A rank sorting can be applied to any content that involves a scale. While most often rankings are based on numeric scores, they can also be used with qualitative scales such as good, better, and best. Sorts based on qualitative ranking are known as enumerations.

Prioritizing by Time: Sorting to Sequence

Much content has a time dimension, and can be compared to other content according to various time frames. These comparisons are made through sequence-related sorting.

Chronological sorts use dates to sort content. They can be valuable in either ascending (earlier to later) or descending (recent to older) order. Ascending chronological order is useful when content items reference and build on each other. For example, I find live blogged stories easier to follow when they are in chronological order, since many statements will depend on what was said in prior statements. Many live blogs choose reverse chronological order, however. The benefit of reverse chronological order is to highlight the newest information, which is assumed to be more important than older information. Reverse chronological order works best when each item is independent of the others, so some live bloggers try to make their statements be understandable without referring to other content items.

An important way to sort content is by how stale or fresh it is. Computer scientists refer to stale content as LRU or “Least Recently Used,” where LRU content is considered obsolete and is purged from the computer’s cache memory. The concept of freshness is captured by a term borrowed from accounting known as LIFO or “Last In, First Out”. Content that is new or changed is generally more valuable than older content. People look for new content, or content that’s been updated.

LIFO is useful as a way to sort behavioral data. In many situations, the content someone most recently viewed will be the most likely content they will want to use again. Imagine someone constantly checking some pages relating to the stock performance of different companies they own or are considering buying. Because they routinely check these pages, the sorting would present these pages as a list ordered according to how recently they were viewed, with the most recently viewed at the top of the list. LIFO behavioral sorting is dynamic, changing as audience interests do, so that new items get added immediately, fading interests soon disappear from the list. It is more flexible, and less work, than having the audience create a custom list.

Sequences are a special kind of list where the order is predefined. Content relating to a sequence should be automatically sorted. Sequences can take various forms:

Procedural with antecedent dependencies, such as “Step 1, Step 2”
Time-defined, such as “Phase 1, Phase 2”
Life-cycle or life-stage based, e.g., “R&D stage, Trials stage, Approval stage, Marketing stage”

A big use of sequences is to position content in time, providing a context for the sorted content. Three kinds of content sequences are:

Before/After sequences
Now/Next sequences
Lower/Higher sequences

Before/After sequences position a topic within a spectrum of time. When items are classified according to their stage, they can be compared. Items at the same stage are similar, and one can locate content about events that preceded that stage, and identify content about other entities that are in a more advanced stage. An example of such a sequence sorting would be articles about a class of new drugs, where different companies are in different stages of market introduction.

Now/Next sequences are similar to Before/After, but are more focused on content relating to a single person or entity, rather than a range of entities. Content can be sorted according to what matches the current context, and list content that will be relevant to the subsequent stage. An example of Now/Next sorting is content about repairing a product.

Lower/Higher sequences define time in terms of proficiency required. The content is ranked according to the abilities of the reader. Publishers frequently classify content according to the level of expertise needed to understand the content. Content might be classified as Beginner, Basic, Intermediate, Advanced, or Expert. The sequence associated with those labels is generally easy to understand. Alternatively, the publisher could rank the difficulty of the content according to a color code:

White belt (= Beginner)
Yellow belt (= Basic)
Blue belt (= Intermediate)
Purple belt (= Advanced)
Black belt (= Expert)

Such rankings can be useful provided the audience knows the level they are at. They might graduate from one tier after viewing all content in that tier. The content sorting might identify and show all Yellow belt content that the reader has not yet seen.

Prioritizing by Novelty: Sorting to Sample

While sorting is generally thought of as involving either ascending or descending order, publishers can also sort items in a statistically random order. This ensures that items presented are unique each time a page loads.

Random sort offered by a content management system. Source: Webflow

For audiences, random sorting provides novelty, offering something that may not have been encountered previously. Random sorting and selection can make content viewing more interesting provided the item pool represents content of potential interest.

Random presentation of content can be used to discover if certain content receives greater than expected attention. Publishers might discover through random promotion of content that audiences are interested in topics that previously did not receive much attention when prioritized by frequency of views.

Prioritizing by Relationships: Sorting to Profile and Evaluate

Sometimes audiences need to sort content to see the relationships between items. One example would be to see what’s more general and what’s more specific. For example, Wikipedia uses a hierarchy based on categories, subcategories, and pages. The entry for a category will be broader in scope than an entry for a page that’s not a category.

For many topics, audiences understand which items are broader than others. But for more specialized fields, automated sorting of topics from broader to narrower is useful. Suppose someone encountered content relating to enzymes. They see content on the following topics:

Acid preparations
Digestives
Betaine hydrochloride

The list has no order. Unless the reader is a specialist, they would not know which topic is the most specific. The sorting order from broader to narrower would be Digestives > Acid preparations > Betaine hydrochloride. Hierarchies provide context for the content, indicating what is background and what is detail.

Websites typically present hierarchies to audiences as an input into a task to complete. The website will require audiences to assess visual relationships in a menu, and select the narrower option in a manual process of “drilling down”. Alternatively, publishers can incorporate such sorting into an automated presentation of content, where the content order is predetermined for the audience. Two common patterns are to show broader topics followed by narrower examples, and to show a narrow example then present the broader topic it represents. Instructional content often utilizes non-visible, programmatic hierarchies to guide presentation of content, to ensure comprehension, or to encourage the review of key concepts.

Nested sorts involve sorting on two or more criteria, such as sorting on a qualitative category and a numeric value together. They are difficult for users to specify themselves, so are best offered pre-packaged. Nested sorts are useful for dynamic content that will change frequently. PolitiFact, the Pulitzer-Prize winning website, rates statements by politicians according to their veracity. A nested sort allows the audience to see how many statements by a politician were associated with different categories of truthfulness:

True
Mostly True
Half-True
Mostly False
False
Pants on Fire!

Nested sort of statements made by Tim Kane. Source: PolitiFact

Automated Editorial Sorting

Content designers and content engineers should consider how computers can order lists to deliver the greatest audience benefits. This approach can be described as automated editorial sorting. It is automated, in the sense that a computer algorithm performs the sort without requiring user interaction. And it is editorial in the sense that prioritization delivered by the sort reflects a judgment concerning what information is most valuable to highlight.

Sorting should not be treated as generic functionality that can be applied indiscriminately to any kind of content. Sorting should provide context for audiences. To be valuable, sorting should surface the content that audiences consider to be their highest priority. It is not enough to give audiences tools to dig out that information themselves. Audiences expect publishers to anticipate what they need, and present content to them in a well ordered manner.

— Michael Andrews

Tags prioritization

Content Engineering

Format Free Content and Format Agility

Post author By Michael Andrews
Post date March 20, 2016
1 Comment on Format Free Content and Format Agility

A core pillar supporting the goal of reusable modules of content is that the content should be “format free”. Format free conveys a target for content to attain, but the phrase tends to downplay how readily content can be transformed from one state to another. It can conceal how people need to receive content, and whether the underlying content can support those needs.

I want to bring the user perspective into the discussion of formats. Rather than only think about the desirability of format neutrality, I believe we should broaden the objective to consider the concept of format readiness. Instead of just trying to transcend formats, content engineers should also consider how to enable customized formats to support different scenarios of use. Users need content to have format flexibility, a quality that doesn’t happen automatically. Not all content is equally ready for different format needs.

The Promise and Tyranny of Formats

Formats promise us access to content where we want it, how we want it. Consider two trends underway in the world of audio content. First, there is growing emphasis on audio content for in-car experiences. Since staring at a screen while driving is not recommended, auto makers are exploring how to make the driving experience more enriching with audio content. A second trend goes in the opposite direction. We see a renewed interested in a nearly dead format, the long playing record disc, with its expressive analog sensuality. Suddenly LPs are everywhere, even in the supermarket. The natural progression of these trends is that people buy a record in the supermarket, and then play the record in their car as soon as they reach the parking lot. An enveloping sonic experience awaits.

Playing records in your car may sound far fetched. But the idea has a long pedigree. As Consumer Reports notes: “A new technology came on the market in the mid-1950s and early 1960s that freed drivers from commercials and unreliable broadcast signals, allowing them to be the masters of their motoring soundtrack with their favorite pressed vinyl spinning on a record player mounted under the dash.”

Highway Hi-Fi record player. Image via Wikipedia.

In 1956, Chrysler introduced Highway Hi-Fi, an in-dash record player that played special sized discs that ran at 16 ⅔ rpms — half the speed of regular LPs, packing twice the playtime. You could get a Dodge or DeSoto with a Highway Hi-Fi, and play records such as the musical the “Pajama Game.” The Highway Hi-Fi came endorsed by the accordion playing taste maker, Laurence Welk.

Sadly playing records while driving in your car didn’t turn out to be a good idea. Surprise: the records skipped in real-world driving conditions. Owners complained, and Chrysler discontinued the Highway Hi-Fi in 1959. Some hapless people were stuck with discs of the Pajama Game that they couldn’t play in their cars, and few home stereos supported 16 ⅔ play. The content was locked in a dead format.

Format Free and Transcending Limitations

Many people imagine we’ve solved the straight jacket of formats in the digital era. All content is now just a stream of zeros and ones. Nearly any kind of digital content can be reduced to an XML representation. Format free implies we can keep content in a raw state, unfettered by complicating configurations.

Format free content is a fantastic idea, worth pursuing as far as possible. The prospect of freedom from formats can lead one to believe that formats are of secondary importance, and that content can maintain meaning completely independently of them.

The vexing reality is that content can never be completely output-agnostic. Even when content is not stored in an audience-facing format, that doesn’t imply it can be successfully delivered to any audience-facing format. Computer servers are happy to store zeros and ones, but humans need that content translated into a form that is meaningful to them. And the form does ultimately influence the substance of the content. The content is more the file that stores it.

Four Types of Formats

In many cases when content strategists talk about format free content, they are referring to content that doesn’t contain styling. But formats may refer to any one of four different dimensions:

The file format, such as whether the content is HTML or PDF
The media format, such as whether the content is audio, video, or image
The output format, such as whether the content is a slide, an article, or a book
The rendered formatting, or how the content is laid out and presented.

Each of these dimensions impacts how content is consumed, and each has implications for what information is conveyed. Formats aren’t neutral. One shouldn’t presume parity between formats. Formats embody biases that skew how information is conveyed. Content can’t simply be converted from one format to another and express the content in the same way.

Just Words: The Limitations of Fixed Wording

Let’s start with words. Historically, the word has existed in two forms: the spoken word, and the written word. People told stories or gave speeches to audiences. Some of these stories and speeches were written down. People also composed writings that were published. These writings were sometimes read aloud, especially in the days when books were scarce.

Today moving between text and audio is simple. Text can be synthesized into speech, and speech can be digitally processed into text. Words seemingly are free now from the constraints of formats.

But converting words between writing and speech is more than a technical problem. Our brains process words heard, and words read, differently. When reading, we skim ahead, and reread text seen already. When listening, we need to follow the pace of the spoken word, and require redundancy to make sure we’ve heard things correctly.

People who write for radio know that writing for the ear is different from writing for a reader. The same text will not be equally effective as audio and as writing. National Public Radio, in their guidebook Sound Reporting, notes: “A reader who becomes confused at any point in [a] sentence or elsewhere in the story can just go back and reread it — or even jump ahead a few paragraphs to search for more details. But if a listener doesn’t catch a fact the first time around, it’s lost.” They go on to say that even the syntax, grammar and wording used may need to be different when writing for the ear.

The media involved changes what’s required of words. Consider a recipe for a dish. Presented in writing, the recipe follows a standard structure, listing ingredients and steps. Presented on television, a recipe follows a different structure. According to the Recipe Writers Handbook, a recipe for television is “a success when it works visually, not when it is well written in a literary, stylistic, or even culinary sense.” The book notes that on television: “you must show, not tell; i.e., stir, fry, serve…usually under four minutes.” Actions replace explicit words. If one were to transcribe the audio of the TV show, it is unlikely the text would convey adequately how to prepare the dish.

The Hidden Semantics of Presentational Rendering

For written text, content strategists prudently advise content creators to separate the structure of content from how it is presented. The advice is sensible for many reasons: it allows publishers to restyle content, and to change how it is rendered on different devices. Cascading Style Sheets (CSS), and Responsive Web Design (RWD) frameworks, allow the same content to appear in different ways on different devices.

Restyling written content is generally easy to do, and can be sophisticated as well. But the variety of CSS classes that can be created for styling can overshadow how rudimentary the underlying structures are that define the meaning of the text. Most digital text relies on the basic structural elements available in HTML. The major elements are headings at different levels, ordered and unordered lists, and tables. Less common elements include block quotes and code blocks. Syntaxes such as Markdown have emerged to specify text structure without presentational formatting.

While these structural elements are useful, for complex text they are not very sophisticated. Consider the case of a multi-paragraph list. I’m writing a book where I want to list items in a series of numbered statements. Each numbered statement has an associated paragraph providing elaboration. To associate the explanatory paragraph with the statement, I must use indenting to draw a connection between the two. This is essentially a hack, because HTML does not have a concept of an ordered list item elaboration paragraph. Instead, I rely on pseudo-structure.

When rendered visually, the connection between the statement and elaboration is clear. But the connection is implicit rather than explicit. To access only the statement without the elaboration paragraph, one would need to know the structure of the document beforehand, and filter it using an XPath query.

Output Containers May Be Inelastic

Output formats inform the structure of content needed. In an ideal world, a body of structured content can be sent to many different forms of output. There’s a nifty software program called Pandoc that lets you convert text between different output formats. A file can become an HTML webpage, or an EPUB book, or a slide show using Slidy or DZSlides.

HTML content can be displayed in many containers. But those containers may be of vastly different scales. Web pages don’t roll up into a book without first planning a structure to match the target output format. Books can’t be broken down into a slide show. Because output formats inform structure required, changing the output format can necessitate a restructuring of content.

The output format can effect the fidelity of the content. The edges of a wide screen video are chopped off when displayed within the boxy frame of an in-flight entertainment screen. We trust that this possibility was planned for, and that nothing important is lost in the truncated screen. But information is lost.

The Challenges of Cross-Media Content Translation

If content could be genuinely format free, then content could easily morph between different kinds of media. Yet the translational subtleties of switching between written text and spoken audio content demonstrate how the form of content carries implicit sensory and perceptual expectations.

Broadly speaking, five forms of digital media exist:

Text
Image
Audio
Video
Interactive.

Video and interactive content are widely considered “richer” than text, images and audio. Richer content conveys more information. Switching between media formats involves either extracting content from a richer format into a simpler one, or compiling richer format content using simpler format inputs.

The transformation possibilities between media formats determine:

how much automation is possible
how usable the content will be.

From a technical perspective, content can be transformed between media as follows.

Media format conversion is possible between text and spoken audio. While bi-directional, the conversion involves some potential loss of expressiveness and usability. The issues become far more complex when there are several speakers, or when non-verbal audio is also involved.

Various content can be extracted from video. Text (either on-screen text, or converted from spoken words in audio) can be extracted, as well as images (frames) and audio (soundtracks). Machine learning technologies are making such extraction more sophisticated, as millions of us answer image recognition CAPTCHA quizzes on Google and elsewhere. Because the extracted content is divorced from its context, its complete meaning is not always clear.

Transforming interactive content typically involves converting it into a linear time sequence. A series of interactive content explorations can be recorded as a non-interactive animation (video).

Simple media formats can be assembled into richer ones. Text, images and audio can be combined to feed into video content. Software exists that can “auto-create” a video by combining text with related images to produce a narrated slide show. From a technical perspective, the instant video is impressive, because little pre-planning is required. But the user experience of the video is poor, with the content feeling empty and wooden.

Interactive content is assembled from various inputs: video, text/data, images, and audio formats. Because the user is defining what to view, the interaction between formats needs to be planned. The possible combinations are determined by the modularity of the inputs, and how well-defined they are in terms of metadata description.

translation of content between formats — Translation of content between formats

Atomic Content Fidelity

Formats of all kinds (file, output, rendering, and media) together produce the form of the content that determines the content experience and the content’s usability.

File formats can influence the perceptual richness (e.g., a 4k video verses a YouTube-quality one).
Rendition formatting influences audience awareness of distinct content elements.
Output formats influence the pacing of how content gets delivered, and how immersive content the content engagement will be.
Media formats influence how content is processed cognitively and emotionally by audiences and viewers.

Formats define the fidelity of the content that conveys the intent behind the communication. Automation can convert formats, but conversion won’t necessarily preserve fidelity.

Format conversions are easy or complex according to how the conversion impacts the fidelity of the content. Let’s consider each kind of content format in turn.

File format conversions are easy to do, and any loss in fidelity is generally manageable.

Rendition format conversions such as CSS changes or RWD alternative views are simple to implement. In many cases the impact on users is minimal, though in some cases contextual content cues can be lost in the conversion, especially when a change in emphasis occurs in what content is displayed or how it is prioritized.

Output format conversion is tricky to do. Few people want to read an e-book novel on their Apple Watch. The hurdles to automation are apparent when one looks at the auto-summarization of a text. Can we trust the software to identify the most important points? An inherent tension exists between introducing structures to control machine prioritization of content, and creating a natural content flow necessary for a good content experience. The first sentence of a paragraph will often introduce the topic and main point, but won’t always.

Media format conversion is typically lossy. Extracting content from a rich media format to a simpler one generally involves a loss of information. The automated assembly of content rich media formats from content in simpler formats often feels less interesting and enjoyable than rich formats that were purposively designed by humans.

Format Agility and Content as Objects

We want to transcend the limitations of specific formats to support different scenarios. We also want to leverage the power of formats to deliver the best content experience possible across different scenarios. One approach to achieve these goals would be to extend some of the scenario-driven, rules-based thinking that underpins CSS and RWD, and apply it more generally to scenarios beyond basic web content delivery. Such an approach would consider how formats need to adjust based on contextual factors.

If content cannot always be free from the shaping influence of format, we can at least aim to make formats more agile. A BBC research program is doing exciting work in this area, developing an approach called Object Based Media (OBM) or Object Based Broadcasting. I will highlight some interesting ideas from the OBM program, based on my understanding of it reading the BBC’s research blog.

Object-Based Media brings intelligence to content form. Instead of considering formats as all equivalent, and independent of the content, OBM considers formats in part of the content hierarchy. Object Based Media takes a core set of content, and then augments the content with auxiliary forms that might be useful in various scenarios. Content form becomes a progressive enhancement opportunity. Auxiliary content could be subtitles and audio transcripts that can be used in combination with, or in leu of, the primary content in different scenarios.

During design explorations with the OBM concept, the BBC found that “stories can’t yet be fully portable across formats — the same story needed to be tailored differently on each prototype.” The notion of tailoring content to suit the format is one of the main areas under investigation.

A key concept in Object-Based Media is unbundling different inputs to allow them to be configured in different format variations on delivery. The reconfiguration can be done automatically (adaptively), or via user selection. For example, OBM can enable a video to be replaced with an image having text captions in a low bandwidth situation. Video inputs (text, background graphics, motion overlays) are assembled on delivery, to accommodate different output formats and rendering requirements. In another scenario, a presenter in a video can be replaced with a signer for someone who is hearing impaired.

The BBC refers to OBM as “adjustable content.” They are looking at ways to allow listeners to specify how long they want to listen to a program, and give audiences control over video and audio options during live events.

Format Intelligence

In recent years we’ve witnessed remarkable progress transcending the past limitations that formats pose to content. File formats are more open, and metadata standards have introduced more consistency in how content is structured. Technical progress has enabled basic translation of content between media formats.

Against this progress taming idiosyncrasies that formats pose, new challenges have emerged. Output formats keep getting more diverse: whether wearables or immersive environments including virtual reality. The fastest growing forms of content media are video and audio, which are less malleable than text. Users increasingly want to personalize the content experience, which includes dimensions relating to the form of content.

We are in the early days of thinking about flexibility in formats that give users more control over their content experience — adjustable content. The concept of content modularity should be broadened to consider not only chunks of information, but chunks of experience. Users want the right content, at the right time, in the right format for their needs and preferences.

— Michael Andrews

Tags formats