Search engines and AI systems have been inextricable for a while now, but with the lines between search engines and AI chatbots beginning to blur, it’s more important than ever that web content satisfies both the user (as per Google’s Helpful Content update) and the AI systems responsible for indexing and recalling it.
The first half of this challenge is easy - as human beings, we know what other human beings want and need - but determining what it is exactly that AI bots look for in rank-worthy content is a different kettle of fish.
This article delves into the dynamic synergy between AI and SEO, focusing on the specific features that wield significant influence over the AI reception of your web pages.
NLP SEO
To understand what AI systems look for in your content, it’s first important to recognise how they’re looking at it — Natural Language Processing (NLP).
NLP is a branch of artificial intelligence that enables machines to comprehend human language. NLP AI systems can interpret the meaning behind words, phrases, and sentences, and process content similarly to humans (kind of), making it possible for bots to assess the relevance and quality of content.
This framework is highly effective but by no means matches human parsing of language, so you may consider taking steps to make your content more NLP-friendly:
1 - Incorporate search intent in your keyword research
The intent of a user’s inquiry informs Google and other AI bots’ gathering process when assembling SERPs. By thinking ahead and composing content that satisfies particular search intents, you highlight the page’s relevance while making it easier for AI to index it correctly.
2 - Prioritise clarity
If you were talking to someone who wasn’t fluent in your native language, you’d speak simply and clearly to help them extract essential information from the exchange — it makes sense to do the same for AI bots.
It’s good practice to use rudimentary, single-idea sentences, and to use clear organised structures that make proper use of simple HTML markup such as H1, H2, and so on.
3 - Research relevant entities
When Google is giving your page a once over, it’s trying to find “entities” — entities are defined as distinguishable things or concepts.
With the various entities in your content labelled, Google uses corresponding descriptors and attributes to connect those that are semantically related and form a holistic understanding of the content, but this is just the tip of the iceberg.
Google is also trying to match each entity in your content to entities filed in something known as the Knowledge Graph, a vast informational resource that Google draws upon to present info boxes besides search results.
It’s also an encyclopaedic database of things and concepts that Google is well acquainted with.
Smoothing out Google’s connect-the-dots style semantic parsing is a matter of identifying and using well-established entities from the Knowledge Graph within your content and structured language — this is also known as ontology alignment.
Now, the keen-eyed SEOs reading this are no doubt thinking that this just sounds like keyword research, and in essence, it is, but with a slightly different scope.
Typical keyword research is about finding popular search terms, but entity research focuses exclusively on context and helping Google understand your content.
Advanced AI bots utilise an almost identical process to form their own databases, but that’s not to say the knowledge graph of an AI bot will be a mirror image of Google’s — optimising for different AI systems may require a two-pronged ontology engineering approach.
Structured data & schema markup
Remember when we said that NLP enables AI to analyse content almost as if it were a human? Well, the operative word here is almost.
AI content analysis falls short of human analysis similarly to how AI-produced content falls short of organically produced content — it’s nearly there, but there are still a few flaws in execution.
This is why it’s good practice to offer our AI friends a helping hand, and we can do so by using structured data and schema markup.
For the uninitiated — structured data & schema markup is a language that machines can understand. Whereas there is great potential for a machine to misinterpret content designed for a human audience, AI can make perfect sense of well-crafted structured data in an instant with zero confusion.
By explaining the nature of a webpage in the native language of the machine, you ensure that it’s able to note the relevance of the page, that information cannot be misunderstood, and that the page will be properly indexed by the bot. This increases the likelihood of superior visibility on the SERPs.
Choosing your structured data language
Structured data isn’t a single language, but in this context, three: RDFa, Microdata, and JSON-LD.
Most consider JSON-LD to be the best of the bunch because, as a scripting language, it’s much easier to use than the HTML-based RDFa and Microdata languages.
Users who aren’t too familiar with Linked Data technology or inline markup can simply populate templates with the necessary JSON-LD data values, and voilà — AI bots can read their page in a flash.
This fluent reading ensures that AI bots understand the core concepts of your content, that they can glean its quality and value, and that it will be appropriately indexed, leading to enhanced visibility on the SERPs.
JSON-LD also scales well and can be added, updated, or removed from sites without much hassle, which, in turn, reduces the chances of human error during maintenance. This adaptability makes it particularly useful for dynamic content that sees a lot of change, such as product data.
APIs & AI
API stands for Application Programming Interface, a name given to software that facilitates open communication between two discrete applications.
Most websites incorporate numerous apps in order to improve the functionality of their pages, and APIs stitch all these disparate elements together and provide a flow of information that enhances the usability and technical performance of a page.
This is fantastic for SEO in of itself, but APIs can also solve a particular problem for Google — a problem involving SPAs.
SPAs (Single Page Applications) handle most user interactions dynamically, meaning all actions play out on the page itself, facilitated by a fluid line of communication between JavaScript and Web APIs.
This is great for users, as it makes for a swift and engaging event, but it can pose a hurdle for crawling and indexing systems that aren’t entirely privy to the exchange.
AI systems don’t always know what to make of SPAs, but prerendering or server-side rendering APIs can simplify the situation and remove doubt by enabling search engines to process SPAs as though they were garden-variety static pages.
It’s important to note that APIs themselves don’t impact the ability of search engines and AI bots to crawl your site, but generally speaking, utilising APIs that help AI bots process site data in a structured and familiar way will improve the accuracy of indexing and increase site visibility.
Performance optimisation
If there’s anything that users and AI bots can agree on, it’s that good page performance is absolutely indispensable.
Seeing that Google has trained its AI systems to appreciate aspects of a page or content that bring value to the user, it's easy to understand why it considers swift page load times and other low latency events important ranking factors.
To give you an idea of the sort of performance you should be aiming for, the listings on the first page of Google typically have an average page speed of 1.65 seconds — and don’t forget to optimise for mobile users too!
Approximately 60% of all online searches occur on mobile devices, so taking steps to maximise mobile responsiveness is crucial if you want AI to notice your site.
Content quality
Google’s golden rule is that content should provide value to users, and as mentioned earlier, with the help of some structured data, NLP allows AI to assess content for quality. This means that creating and maintaining high-value content should always be one of your top priorities where SEO is concerned.
On a base level, “high value” means that your content should be well-structured and accurate, but on an elevated level, it needs to solve a problem for your user. In other words, it should recognise the intent of the user’s search and provide the information they’re looking for in a digestible and engaging manner.
Wherever possible, you should also endeavour to offer the user something unique in your content, i.e., multimedia elements such as images, videos or transcripts, but bear in mind that AI systems may not understand these additions.
To ensure bots get the multimedia memo, you’ll need to label them sufficiently with alternative text — written descriptions of non-textual elements on your webpage.
The primary objective of alt text is to make web pages more accessible by providing an alternative means of communicating multimedia that, otherwise, would be imperceivable to differently-abled users.
It just so happens that these small but mighty descriptions also help AI bots understand the full impact of your webpage when it contains more than text-based content.
Content accessibility
Alt text isn’t the only way to make your web page more accessible to a wider user audience.
WCAG
Website Content Accessibility Guidelines (WCAG) provide a comprehensive list of best practices for boosting the inclusivity of your content, and many of them also provide greater clarity to AI bots.
The WCAG are built on four key principles of accessibility: Your content should be perceivable, operable, understandable, and robust. Following the guidance provided by this authority on achieving these four states is a surefire way to improve page usability for humans and readability for AI bots.
ARIA
ARIA (Accessible Rich Internet Applications) is another set of guidelines and technologies used to make websites and web applications more accessible to differently-abled users.
ARIA consists of markup that can optimise HTML to provide additional information to assistive technologies. For example, it can tell a screen reader that a button is a "close" button, or that a certain area of a web page is a navigation menu. This extra information helps users with disabilities know what's on a page and how to interact with it.
While there is no direct link between the implementation of ARIA accessibility guidelines and SEO, there are several beneficial indirect links, such as improved user experience, increased traffic and accessibility compliance.
Selecting relevant keywords in your ARIA labels can also help to reinforce your primary keyword efforts and make page elements more legible to Google and other search engines.
Crawlability
Using ‘robots.txt’ files and XML sitemaps, you can guide AI bots through the pages of your website.
This is of particular importance if you’re hoping to optimise your pages for AI bots, as their crawling facilities are far inferior to Google’s, and until they reach the same degree of sophistication, they’ll run into problems when crawling your site.
Deep linking is another crucial element of crawlability to focus on — links are the portals through which AI bots navigate your website, just like human users.
By strategically threading internal links that lead to all essential pages on your site, you guarantee that AI bots can crawl and index them effectively. Orphan pages, on the other hand, will not be accessed by crawlers and will remain uncategorised and effectively invisible.
Bot traffic analytics
As they aren’t as transparent with their needs as humans are, SEO for AI bots can sometimes feel like a stab in the dark. However, with certain analytics tools, you can monitor bot traffic and how bots interact with your content in order to gain insights into what they’re looking for.
Even advanced analytics tools have trouble monitoring all bot traffic your site receives, as some bots hide behind different user agents or spoofing, but you’ll be able to draw out enough data from the bots that do get monitored to build an idea of their general focus.
Once identified, you can optimise points of AI interest on your site and monitor site performance after subsequent visits from bots to see if your efforts are paying off.
Final thoughts
You likely noticed that there’s some significant overlap between what AI and human users are looking for in a valuable web page, but it’s good to develop an understanding of how and when these elements of SEO relate.
By learning how AI-powered elements shape the digital marketing landscape, we can better equip ourselves to navigate the intricate web of algorithms that determine online visibility. In turn, this enables us to get ahead of AI’s seismic impact on SEO and how consumers access content, placing us in a position to orchestrate your digital presence for success.
Contact us if you’d like to put the theory discussed here into practice and use SEO to optimise your site for AI.