Getting our Content Ready for the Spoken Word

More of us are starting to talk to the Internet and listen to it, in lieu of tapping our screens and keyboards and reading its endless flow of words. We are changing our relationship with content. We expect it to be articulate. Unfortunately, it sometimes isn’t.

Tired of Text

To understand why people want to hear text, it is useful to consider how they feel about it on the screen. Many people feel that constant reading and writing is tiring. People from all walks of life say they feel spelling-challenged. Few people these days boast of being good spellers. Spelling can be a chore that often gets in the way of us getting what we want. Many basic web tasks assume you’ll type words to make a request, and a computer depends on you to tap that request correctly.

Spelling is hard. According the to American Heritage Dictionary, in standard English, the sound of the letter k (as in “kite”) can be represented 11 ways:

• c (call, ecstasy)
• cc (account)
• cch (saccharin)
• ch (chorus)
• ck (acknowledge)
• cqu (lacquer)
• cu (biscuit)
• lk (talk)
• q (Iraqi)
• qu (quay)
• que (plaque)

Other sounds have equally diverse ways of being written. If we factor in various foreign words and made-up words, the challenge of spelling feels onerous. And the number of distinct written words we encounter, from people’s names to product names, grows each year. Branding scholars L. J. Shrum and Tina M. Lowrey note the extent to which brands go to sound unique: “There are also quite a number of nonobvious, nonsemantic ways in which words can convey both meaning and distinctiveness. Some examples include phonetic devices such as rhyming, vowel repetition, and alliteration, orthographic devices such as unusual spellings or abbreviations, and morphological devices such as the compounding or blending of words.” This “distinctiveness” creates problems for people trying to enter these names in a search box.

The Writing – Speaking Disconnect

People face two challenges: they may not know how to say what they read, or how to write what they hear.

People are less confident of their spelling as they become more reliant on predictive text and spellchecking.

Readers encounter a growing range of words that are spelled in ways not conforming to normal English orthography. For example, a growing number of brand names are leaving out vowels or are using unusual combinations of consonants to appear unique. Readers have trouble pronouncing the trademarks, and do not know how to spell them.

A parallel phenomenon is occurring with personal names. People, both famous and ordinary, are adopting names with unusual spellings or nonconventional pronunciations to make their names more distinctive.

As spelling gets more complicated, voice search tools such as Google Voice, Apple Siri, Microsoft Cortana and the Amazon Echo are gaining in popularity. Dictating messages using Text-to-Speech synthesis is becoming commonplace. Voice-content interaction changes our assumptions about how content needs to be represented. At present, voice synthesis and speech recognition are not up to the task of dealing with unusual names. Android, for example, allows you to add a phonetic name to a contact to improve the ability to match a spoken name. Facebook recently added a feature to allow users to add phonetic pronunciation to their names. [1] These developments suggest that the phonetic representation of words is becoming an increasingly important issue.

The Need for Content that Understands Pronunciation

These developments have practical consequences. People may be unable to find your company or your product if it has an unusual name. They may be unable to spell it for a search engine. They may become frustrated when trying to interact with your brand using a voice interface. Different people may pronounce your product or company name in different ways, causing some confusion.

How can we make people less reliant on their ability to spell correctly? Unfortunately there does not seem to be a simple remedy. We can however learn from different approaches that are used in content technology to determine how we might improve the experience. Let’s look at three areas:

• Phonetic search
• Voice search
• Speech synthesis

Phonetic Search

Most search still relies on someone typing a query. They may use auto-suggest or predictive text, but they still need to know how something is spelled to know if the query written matches what they intend.

Phonetic search allows a user to search according to what a word sounds like. It’s a long established technology but is not well known. Google does not support it, and consequently SEO consultants seldom mention it. Only one general purpose search engine (Exalead, from France’s Dassault Systèmes) supports the ability to search on words according to what they “sound like.” It is most commonly seen in vertical search applications focused on products, trademarks, and proper names.

To provide results that match sounds instead of spelling, the search engine needs to be in the mode of phonetic search. The process is fairly simple. The search engine identifies the underlying sounds represented by the query and matches it with homonyms or near-homonyms for that sound. Both the query word and target word are translated into a phonic representation, and when those are the same, a match is returned.

The original form of phonetic search is called Soundex. It predates computers. I first became aware of Soundex on a visit several years ago to the US National Archives in Washington DC. I saw an exhibit on immigration that featured old census records. The census recorded surnames according to the Soundex algorithm. When immigrants arrived in United States, their name might not be spelled properly when written down. Or they may have changed the spelling of their name at a later time. This mutation in the spelling of surnames created record keeping problems. Soundex resolves this problem by recording the underlying phonetic sound of the surname, so that different variants that sounded alike could be related to one another.

The basic idea behind Soundex is to strip out vowels and extraneous consonants, and equalize similar-sounding and potentially confused consonants (so that m and n are encoded the same way). Stressing the core features of the pronunciation reduces the amount of noise in the word that could be caused by mishearing or misspelling. People can use Soundex to do genealogical research to identify relatives who changed the spelling of their names. My surname “Andrews” is represented as A–536, which is the same as someone with the surname of “Anderson.”[2]

Soundex is very basic and limited in the range of word sounds it can represent. But it is also significant because it is used in most major relational database software such as Oracle and MySQL. Newer, NoSql databases such as elasticsearch, also support phonetic search. Newer, more sophisticated phonetic algorithms offer greater specificity and can represent a wider range of sounds. But by broadening the recall of items, it will decrease the precision of these results. Accordingly, phonetic search should only be used selectively for special cases targeting words that are both often-confused and often-sought.

An example of phonetic search is available from the databases of the World Intellectual Property Organization (WIPO), a unit of the United Nations. I can do a phonetic search of a name to see what other trademarks sound like it. This is an important issue, since the sound of a name is an important characteristic of a brand. Many brand names use Latin or Greek roots and can often sound similar.

Let’s suppose I’m interested in a brand called “XROS.” I want to know what other brands sound like XROS. I enter XROS in WIPO’s phonetic search, and get back a list of trademarks that sound similar. These include:

• Sears
• Ceres
• Sirius
• XROSS
• Saurus

Phonetic search provides results not available from fuzzy string matching. Because so many different letters and letter combinations can represent sounds, fuzzy string matching can’t identify many homonyms. Phonetic search can allow you to search for names that sound similar but are spelled differently. A search for “Smythe” yields results for “Smith.” An interesting question arises when people search for a non-word (a misspelled word) that they think sounds like the target word they seek. In the Exalead engine, there is a different mode for spellslike compared with soundslike. I will return to this issue shortly.

Voice Search

With voice search, people expect computers to worry about how a word is spelled. It is far easier to say the word “flicker” and get the photo site Flickr than it is to remember the exact spelling.

Computers however do not always match the proper word when doing a voice search. Voice search works best for common words, not unique ones. As a consequence, voice search will typically return the most common close match rather than the exact match. To deal with homonyms, voice search relies on predictive word matching.

The challenge voice search faces is most apparent when it tries to recognize people’s names, or less common brand names.

Consider the case of “notable” names: the kind that appear in Wikipedia. Many Wikipedia entries have a phonetic pronunciation guide. I do not know if these are included in Google’s knowledge graph or not, but if they are, the outcomes do not seem consistent. Some voice searches for proprietary names work fine, but others fail terribly. A Google voice search for Xobni, a email management tool bought by Yahoo, provides results for Daphne, the Greek mythological goddess.

Many speech recognition applications use an XML schema called the Pronunciation Lexicon Specification (PLS), a W3C standard. These involve what is called a “lexicon file” written in the Pronunciation Lexicon Markup Language (an XML file with the extension of .pls) that contains pronunciation information that is portable across different applications.

A Microsoft website explains you can use a lexicon file for “words that feature unusual spelling or atypical pronunciation of familiar spellings.” It notes: “you can add proper nouns, such as place names and business names, or words that are specific to specialized areas of business, education, or medicine.” So it would seem ideal to represent the pronunciation of brands’ trademarks, jargon, and key personnel.

The lexicon file consists of three parts: the <lexeme> container, and the <grapheme> (word as spelled) and <phoneme> (word as pronounced.) The schema is not complicated, though there is a little effort to translate a sound represented in the phoneme into the International Phonetic Alphabet, which in turn must be represented in a character-set XML recognizes. A simple dedicated translation tool could help with this task.

While incorporating a lexicon file will not improve visibility on major search engines, these files can be utilized by third party XML-based voice recognition applications from IBM, Microsoft and many others. One can also provide a lexicon profile for specific words in HTML content using the microformat rel=“pronunciation”, though this does not appear to be extensively supported right now. So far, voice search on the web has been a competitive contest between Google/Apple/Amazon/Microsoft to develop the deepest vocabulary. Eventually, voice search may become a commodity, and all parties will want user-supplied assistance to fine-tune their lexicons, just as they do when encouraging publishers to supply schema.org metadata markup.

In summary, digital publishers currently have a limited ability to improve the recall in voice searches of their content on popular search engines. However, the recent moves by Google and Facebook to allow user-defined custom phonetic dictionaries suggests that this situation could change in the future.

Speech Synthesis

Text-to-Speech (TTS) is an area of growing interest as speech synthesis becomes more popular with consumers. TTS is becoming more ubiquitous and less robotic.

Nuance, the voice recognition software company, is focused increasingly on the consumer market. They have new products to allow hands-free interaction such as Dragon TV and Dragon Drive that not only listen to commands, but talk to people. These kinds of developments will increase the desirability of good phonetic representation.

If people have trouble pronouncing your trademarks and other names associated with your brand, it is likely that TTS systems will as well. An increasing number of products have synthetic names or nonstandard names that are difficult to pronounce, or whose correct pronunciation is unclear. FAGE^® Greek Yogurt — how does one pronounce that?[3] Many English speakers would have trouble pronouncing and spelling the name of the world’s third-largest smartphone maker, Xiaomi (小米).[4] As business is increasingly global, executives at corporations often come from non-English-speaking countries and will have foreign names that are unfamiliar to many English speakers. You don’t want a speech synthesis program to mangle the name of your product or the name of your senior executive. One can’t expect speech synthesis programs to correctly pronounce unusual names. Brands need to provide some guidance for voice synthesis applications to pronounce these names correctly.

HTML has standards for speech synthesis: the Speech Synthesis Markup Language (SSML), which provides a means of indicating how to pronounce unusual words. Instructions are included within the <speak> tag. Three major options are available. First, you can indicate pronunciation using the <say-as> element. This is very useful for acronyms: for example do you pronounce the letters as a word, or do you sound out each letter individually? Second, you can use the <phoneme> tag to indicate pronunciation using the International Phonetic Alphabet. Finally, you can link to a XML <lexicon> file described using the Pronunciation Lexicon Markup Language mentioned earlier.

SSML is a long-established W3C standard for Text-to-Speech. While the SSML is the primary way to provide pronunciation guidance for web browsers, an alternate option is available for HTML5 content formatted for EPUB 3, which unlike browser-based HTML, has support for the Pronunciation Lexicon Markup Language.

Making Content Audio-Ready

Best practices to make text-based content audio-ready are still evolving. Even though voice recognition and speech synthesis are intimately related, a good deal of fragmentation still exists in the underlying standards. I will suggest a broad outline of how the different pieces relate to each other.

SSML for speech synthesis provides good support for HTML browser content.

Dedicated voice recognition applications can incorporate the Pronunciation Lexicon Specification’s lexicon files, but there currently is little adoption of this files for general purpose HTML content, outside of ebooks. PLS can (optionally) be used in speech applications in conjunction with SSML. PLS could play a bridging role, but hasn’t yet found that role in the web ecosystem.

Phonetic Search Solutions

The dimension that most lacks standards is for phonetic search. Phonetic search is awkward because it asks the searcher to acknowledge they are probably spelling the term incorrectly. I will suggest a possible approach for internal vertical search applications.

The Simple Knowledge Organization System (SKOS) is a W3C standard for representing a taxonomy. It offers a feature called the hidden label which allows “a character string to be accessible to applications performing text-based indexing and search operations, but would not like that label to be visible otherwise. Hidden labels may for instance be used to include misspelled variants of other lexical labels.” These hidden labels can help match phonetically-influenced search terms with words used in the content.

Rather than ask the searcher to indicate they’re doing a “sounds like” search, it would be better to allow them to find sound-alikes at the same time they are doing a general search. The query form could provide a hint that exact spelling is not required and that they can sound out the word. The search term would look for any matches with the terms in the taxonomy including phonetic equivalents.

Let’s imagine your company has a product with an odd name that’s hard for people to recall. The previous marketing director thought he was clever by naming your financial planning product “Gnough”, pronounced “know” (it rhymes with “dough”!) The name is certainly unique, but it causes two problems. Some people see the word, mispronounce it, and remember their mispronounced version. Others have heard the name (perhaps on your marketing video) but can’t remember how it is spelled. You can include variants for both cases in the hidden labels part of your taxonomy:

• Learned the wrong pronunciation: Include common ways it is mispronounced, such as “ganuff”
• Learned correct pronunciation but can’t spell it: Include common spellings of the pronunciation, such as “no”, “know” or “noh”

The goal is to expand search term matching from simple misspelling that can be caught by fuzzy matching (e.g., transposing letters) to consider phonetic variations (the substitution of a z for a x or s, or common alternatives for representing vowel sounds, for example.) Because increasing search recall will lower search precision, you may want to offer a “did you mean” confirmation showing the presumed term, if there is doubt as to the intention of the searcher.

Prognosis for Articulate Content

Our goal is to make our digital content articulate — intelligible to people when speaking and listening. It is not an easy task, but it is a worthy one.

These approaches are suitable only for a small subset of the vocabulary you use. You should prioritize according to which terms are most likely to be mispronounced or misspelled because of their inherent pronunciation. From this limited list you can then make choices as to how to represent them phonetically in your content.

Articulate content is an especially complex topic because there are many factors outside of one’s immediate control. There are numerous issues of integration. Customers will likely be using many different platforms to interact with your content. These platforms may have proprietary quirks that interfere with standards.

But watch this space. Easy solutions don’t exist right now, but they will likely become easier in the not too distant future — they will need to.

— Michael Andrews