Generative artificial intelligence, LLMs (large language models) and AI search are built upon the shoulders of centuries of accumulated human knowledge and experience. This was made possible by free access to the open web. However, lack of concern for the interests of people who create and own the rights to high quality original content might derail AI projects.
Billions are being invested in a competition to secure technical talent, computing power and energy considered essential to develop the best AI in the world. It is a contest many believe will have few winners and possibly only one. Concerns raised by human content creators are often brushed aside, as are questions regarding ethical, environmental and social issues.
The Growth of The Open Web
For decades people have posted content online, hoping to appeal to their target audience. Some hope to promote their work and build a career, but most do it for fun and to exchange ideas, without expectation of financial gain. Business owners post content intended to promote and sell products and services, often paying people to create content for them.
Search engines were developed to organise and index content on the open web, making it easier for people to find what they wanted. Competition for traffic led to the rise of SEO (search engine optimisation). Search results provide organic and paid links to web pages, which can be monetised by promoting and selling products and services.
An online ecosystem arose that incentivised individual creators, businesses and organisations to put their content on the open web. This has enabled free access to a huge variety of content. Business owners promote products and services on the Internet, using paid advertising links and content that is designed to attract the attention and engage the interest of their potential customers.
The End of The Open Web
For decades the open web has benefited content creators, people seeking content, search engine companies indexing content and businesses marketing and selling products and services. However, if AI search delivers results without the need to visit web pages this removes the incentive for people to create and post original content online. This could lead to the end the open web as we have known it.
Many website owners are finding site resources are being used by AI crawlers that harvest their content, but bring no benefit and increase their costs. This is causing some to put content behind paywalls or block AI, to prevent it accessing their content. AI companies could be sacrificing the proverbial goose that lays golden eggs, in return for short term gain.
In the competition to increase the computing power of AI, access to information on the open web has generally been taken for granted. However, this might soon change. Access to original, high quality, timely content for training and running AI could become a key differentiator between competing AI services. The value of content creators and those who own content rights would greatly increase.
SEO, Content Marketing and The Rise of AI
For many years posting relevant content online and an effective SEO strategy could help a business attract targeted traffic to web pages that market and sell products and services. However, human created content is increasingly being displaced by AI search, AI bots and AI agents. This is disrupting the ability of businesses to attract customers and individuals to earn a living online.
Until recently SERPs (search engine results pages) consisted of organic and paid web page links to content created by people. They competed to rank at or near the top of results pages for relevant keyword searches. However, online search increasingly generates AI results, that deliver little or no traffic to the sites hosting the original source content.
Loss of traffic and income caused by increasing use of AI search is impacting many website owners and content creators. In my article SEO and The Rise of AI Search I explored this from the perspective of SMEs adapting their content marketing strategy to changing online search behaviour, but for content creators the situation appears to be worsening.
The Commoditisation of AI Services
When a new product or service is invented or developed, it often goes though a period of rapid change, during which businesses compete for market shared by highlighting technological differences. From domestic appliances and cars, to computer hardware software and mobile phones, they generally reach a point of maturity, after which change is incremental or superficial.
There are indications that after a period of rapidly growing power, LLMs and generative AI are approaching a plateau, beyond which the cost of further gains will be prohibitive. We could enter a phase in which AI becomes a commodity with standard features and similar levels of performance. Competition could be based on user preferences and benefits rather than technical features.
The quality of AI generated content might decline if it loses access to content on the open web and is restricted to public domain content, or information that it is required to pay for in some manner. If AI is a commodity, not differentiated on a technical performance level, access to original high quality content will be what differentiates them.
In The Age of AI Content Remains King
The content on which AI was trained and on and which it needs to answer user requests was created by human beings. AI can pattern matching and break down the elements of existing content to construct something that simulates human creativity, but it lacks personal perspective and lived experiences. AI cannot create genuinely original work and synthetic AI data is a poor substitute.
Whether created for commercial or non-commercial reasons, content typically has a target audience that content creators wants to reach. The term content is king emerged in the 1990s and while Internet technology has changed, people still want to consume content in its various forms. The best AI models might pay people to create content or licence it from content owners.
There are claims solutions will be found to the issue of so called hallucinations, but AI simply calculates probability of the next character in content it generates. AI hallucinations might be better referred to as errors, caused by lack of access to sufficient relevant content, resulting in predictions that are less reliable and more inaccurate. There might be competition for exclusive rights to domain specific content in attempt to reduce such errors.
Content Fragmentation and AI Specialisation
If access to content becomes more fragmented this could lead to less reliable AI results and a move towards more specialised AI that bases its results on content to which direct access is limited and paid for in some manner. For example, it is possible versions of AI could exist for domains including medicine, business, law, the sciences and creative arts.
Content creators and owners might block access to their information, unless AI services credit and link to original content. Delivering traffic that converts into an income could enable business owners to cover their running costs and earn a profit. This could include links to products and services that a business wants to market and sell.
Some might block AI search, preferring to attract traffic directly to their website for monetisation purposes. In a world where people apply in their life the advice of AI expert services, there might also be a requirement to comply with regulations, such as liability for the consequences of people acting upon inaccurate information.
The Future Is Human
The original vision of the world wide web as a place where people could freely exchange ideas survived many commercial and regulatory changes and regional restrictions. However, the online world is increasingly overwhelmed by a vast quantity of low quality AI generated content, which threatens an economy that supports human content creation.
Some claim AI will outperform human beings at all tasks, radically altering how we live and work. There are predictions that AGI (artificial general intelligence) or ASI (artificial super intelligence) will lead to more change in the next decade than during the previous century. However, it is also possible that we might be approaching, or have already reached, the peak of global AI.
AI can provide useful tools, that assist people with repetitive, time consuming routine tasks. However, innovation and creativity, at which people excel, will ensure the future is human. We might retain access to some version of the open web, but businesses providing AI services will need to recognise, incentivise and reward the value of human content creation if they are to prosper.