The internet is undergoing its most fundamental shift since the rise of the World Wide Web in the 1990s. The shift is so significant that many content professionals don’t appreciate how radically it will change current practices. The webpage is dying, yet what will replace it has yet to be defined.
Organizations have been building web pages for three decades — only a few of us remember the internet before webpages. Webpages are all that most of us have ever known.
AI promises to make building webpages even easier. Some experts imagine AI will trigger an explosion of webpages, increasing their number manyfold. According to this thinking, AI will make it easier to build personalized webpages. We will finally realize the dream of having webpages designed for an audience of one.
That vision is one embraced by developers, for whom building webpages has been the major preoccupation.
But AI isn’t revolutionary because it makes doing the same thing easier. AI is disruptive because it changes user behavior — not developer behavior.
Already, we see evidence that visitor traffic to webpages is down significantly. Users aren’t that into webpages anymore.
It would be a mistake to assume that if webpages became more personalized, people would visit them more often. The website is a declining channel. There is little possibility that it will regain its historic status.
AI bots and agents can provide information more directly than a webpage. Publishers are working on how AI can:
- Answer questions and provide updates
- Book travel or tickets
- Plan tasks
- Find and buy the best product
- Solve customer problems
These topics are the bread and butter of webpages. As attention spans get ever shorter, information must be delivered immediately to be used. Hardly anyone wants to scroll through a webpage anymore if they don’t have to. That’s especially true for users whose expectations have been conditioned by algorithmic feeds such as TikTok.
Some will doubt that webpages will be displaced. They believe that many people prefer webpages over other channels. Or else they believe that webpages will remain necessary.
Sceptics imagine AI bots and agents will be just another channel, and that webpages will remain vital in the future.
In the short run, as the internet undergoes its dramatic transition, we can expect a mixed environment, with AI and webpages coexisting. Organizations will need to support both.
But in the longer term, webpages will become unnecessary for most topics currently addressed by websites. This could happen faster than many people expect.
Three factors will influence how quickly webpages disappear:
- How quickly user behavior shifts to the adoption of AI tools
- How effectively AI bots and agents can address both customer and enterprise tasks
- How readily organizations can support AI bots and agents without creating webpages.
The first two factors are somewhat interrelated. User adoption depends partly on the quality of AI tools in addition to their perceived convenience. The evolution of tools will depend on practicalities relating to AI infrastructure, such as models, orchestration, and ecosystems. While uncertainties remain with both, the amazing strides realized already and phenomenal investments underway suggest that progress on both will continue.
However, the third factor — webpage-free content for AI — remains largely unaddressed.
Unfortunately, the AI engineers developing tools have little expertise in content management. Their tools assume webpages, at least as an initial input. Webpages are published to be crawled, before being tokenized.
But it makes little sense to publish webpages when the webpage’s main audience will be bots seeking to crawl them. It would be more ideal to create content in a format that AI tools can use without needing to convert it.
AI methods and protocols require access to information in machine-readable formats. While they process human-readable text, they also rely on parameters that convey information about the text.
What’s been missing is a definition of what constitutes AI-native content. So far, most efforts have been focused on retrofitting webpage content so that AI tools understand it. Examples include:
- Creating parallel LLM.txt pages
- Adding additional schema.org structured data to webpages to help orient AI bots
- Training bots to understand XML tags, such as the NISO Standards Tag Suite, used to exchange or assemble content that becomes webpages
None of these approaches is truly AI-native, because they still presume the creation of webpages before making content available to AI tools.
Once legacy webpages have been made AI-ready, the focus will shift to how to create new content efficiently — how best to create AI-native content.
Some original content can be database-generated. But much narrative content will still require editorial oversight. Writers will need to decide on the messages to use, the emphasis of information, and the best phrasing. And some new content will be unique in the sense that it isn’t derived from prior content, and must be drafted by humans.
We are still missing the content authoring and management tools that support the development of AI-native content. Human writers need guidance on what bots need so they make good decisions and don’t get confused. Bots require predictability that the information needed to address a question or task is available.
The current approach of creating more webpages and expecting bots to untangle them and find what’s needed won’t be sustainable. Bots are thrown off by duplication. And crunching through repetitive webpages is wastes time, money, and environmental resources.
We have many of the pieces to build an AI-native content development system. (I’m not calling a CMS, since CMSs are intrinsically linked to the website era we are leaving.)
What will be needed is to combine:
- AI editorial writing tools such as prompts, guidelines, and information QA
- Schemas to shape message fragments, roles, descriptions, and provenance information (and also build on the granular structures developed by the translation industry)
- Connection points to protocols for agents and other databases
AI-native content will be very different from the webpage era. Content creators who are comfortable thinking in terms of small pieces will be well-positioned to make the transition.
— Michael Andrews