Machine Learning: bringing SEO and UX Together
Historically SEO’s have been obsessed with link acquisition and its impact on search engine rankings. Fast forward to today, and Google has progressed enormously in the last few years from a relatively static (yet complex) link-based index, to one that is driven by a multitude of technical and user-level factors. And today, the driving force that’s helping Google make sense of this myriad of data points and most importantly, searcher intent, is machine learning.
In this blog post, we discuss some key concepts in relation to the rise of machine learning, optimising for user intent and the very possible (in our humble opinion) decline of link importance as a ranking factor.
The growing importance of UX signals in SEO
There’s a growing consensus and body of evidence to support the idea that Google is using user experience signals to help rank content and determine how useful a piece of content is in relation to a given query. It also seems that with the advent of RankBrain, Google’s ability to do this in near real-time (using signals like Organic CTR and ‘pogo-sticking’) to rank that content even more effectively has stepped up a gear.
With a multitude of human and technical factors feeding into how Google ranks sites, we need to really think about the work that we do and ask ourselves whether or not we're delivering a comprehensive and well-defined user experience - one that enables the user to complete their desired goal - first and foremost.
To do this effectively, I believe there are certain things we can learn from the practice of CRO that can help us get closer to understanding and optimising for user intent, which in turn will help better serve our work as SEOs.
What role does machine learning play in SEO?
It’s been well documented that RankBrain is now the third-largest element in Google’s ranking algorithm, and that it’s primarily concerned with determining the context of search queries it’s never encountered before. Given that 15% of all searches on Google are for queries Google has never seen before, this is a big step forward for Google in understanding the true scope of what searchers really want and need.
Google is also using machine learning and data-driven models to understand increasingly complex human-brand interactions; even enabling them to make scarily accurate predictions around user behaviour. For evidence of this, I highly recommend you read Google and SOASTA’s research into on building a predictive machine-learning algorithm to predict Bounce and Conversion rate.
What are the implications of machine learning for SEO?
So what does it all mean for us marketers? In this new machine-learning age, optimising for the algorithm seems to be a bit like trying to shoot a moving target – something that might well be fruitless. Edmond Lau, former Google Search team engineer, hit the nail on the head:
“It’s hard to explain and ascertain why a particular search result ranks more highly than another result for a given query. It's difficult to directly tweak a machine learning-based system to boost the importance of certain signals over others.”
How do we optimise for constant change?
There are obviously a vast number of technical aspects to modern SEO which I'm not going to discuss here. Suffice to say, these are crucial to success. However, for the purposes of this post, I'm interested in how machine learning is helping Google improve its interpretation of the types of broad terms we as SEO's are throwing around more and more - terms like ‘engagement’ and ‘usefulness’. Secondly, I'm also interested in how we can build upon current understandings around these terms to create more future proof SEO strategies.
What exactly do these terms even mean in Google's eyes? If we dig into the quality-rater guidelines, we can get a pretty good idea. In fact, usefulness is pretty much baked into Google’s entire philosophy for determining content quality. This list, lifted from the quality raters guidelines reads more like a manifesto for good UX than anything, one with usefulness and purpose at the core:
Identify the purpose of the landing page
Identify main content, supplementary content and advertisements. Is it easy to identify main content immediately?
Review main content with regards to the purpose of the page
Determine the amount of useful main content
Determine the benefit of the supplementary content
Searcher satisfaction is crucial
As discussed in Steven Levy's 'In the Plex', perhaps the strongest indicator Google has of a piece of content's quality is the 'long click':
"On the most basic level, Google could see how satisfied users were. To paraphrase Tolstoy, happy users were all the same. The best sign of their happiness was the 'Long Click' — This occurred when someone went to a search result, ideally the top one, and did not return. That meant Google has successfully fulfilled the query."
Building on this qualitative insight, it seems entirely possible Google could be making moves to develop this from an algorithmic point of view, as discussed by Bill Slawski in reference to a patent around the 'long click' filed by Google.
As discussed in the SEMrush Ranking Factors Study 2017, Time On Site, Pages Per Session and Bounce Rate were all found to be positively correlated with high ranking content, suggesting that these user engagment signals bare relevance for SEO. True, these metrics are flawed from a standalone perspective, but their correlation with rankings provides an interesting insight into how the algorithm may currently be intepreting user engagement signals.
“In a post-PageRank world, we’re optimising not only for the click, but for the post-click experience.”
Search intent: bridging UX and SEO
CRO practitioners and UX analysts will be involved in improving a site’s performance in a number of critical ways. This could involve anything from detail user journey mapping, user testing and qualitative analysis to A/B testing.
Irrespective of the techniques used, CRO practitioners are committed to finding out why visitors are behaving in a certain way, not completing a certain action, and fixing the problems causing this block.
As SEO's we're increasingly concerned with optimising for intent and user satisfaction. Therefore there needs to be a stronger dialogue between the SEO, UX and CRO communities; a way to arrive at some similar definitions and conclusions around how to achieve this aim.
Room for collaboration
Call me cynical, but I don’t believe that the SEO and CRO communities have really established a set of shared understandings or definions so far to help us get to a point where we can confidently talk about engagement or searcher satisfsction from an on-page perspective. And my concern is that without that dialogue and collaboration, neither will ever really be able to fully optimise for the click and what happens after the click as the singular experience that is actually is.
It comes down to a fundamental truth about what I'm loosely referring to as the ‘post-Pagerank’ ecosystem – that good user experience and good SEO ought to share the same goals and talk the same language. That all sounds a bit utopian, and possibly far from reality at the moment. Which got me thinking, how might this work in practice? I don't claim to have the answer - but rather I wanted to share a way of thinking that I thought teams could adopt and share to help work towards shared Goals, underpinned by a pretty simple process.
We need a collaborative framework for SEO & CRO
I’ve thought a lot about how the adoption of ‘CRO-thinking’ – the mindset and rigour of optimising around the user and all that encompasses – helps to provide the foundations for this new era of SEO in which optimising for task completion or engagement is critical. In thinking about this, I’ve borrowed some ideas from CRO to try to build a bit of a broad framework which I believe helps, at the very least, to encourage dialogue and collaboration.
This system, can be broadly be defined as following this cyclical process:
1. Goal definition
Where CRO excels as a practice is often in its simplicity. Teams orient around a singular goal, and optimise and measure around that specific goal until a benchmark is met, then they can either move on or reiterate to improve on that benchmark again.
In essence, as SEO's, we need to try to think in similar terms when optimising for user intent. What exactly is it we're trying to do? Do we have a metric in mind that we need to focus on? Do we even have a framework of way of measuring whether we're creating more engaging experiences or not?
Google Analytics is a potent tool, but one that's woefully under-utilised in my opinion. Instead of customising our configurations to focus on what matters, a huge number of marketers still focus on largely flawed and useless vanity metrics like Bounce Rate and Time on Page to evaluate whether a page is engaging or not.
There is so much more we can and should be doing to customise our Analytics configuration to better inform us about what's really important. Instead of focusing on vanity metrics, consider the customer engagement points that matter to you as a site owner or brand, and where these cross over with the user's needs.
Using Google Analytics Goal Value, we can then begin to attribute values to specific micro-interactions and user behaviours that we believe are important to the brand and to the user. By recognising that a user will not always convert on a macro-goal, we can use these micro-interactions better understand how content helps or hinders them in completing what they need to do.
Micro conversions, such as a user navigating to a service page from a blog post, or watching 100% of our brand video help give us a little more of a holistic view of what’s working and what isn’t, which we can then optimise and improve. It’s a simple-to-implement and scalable approach to defining meaningful Goals that we can use to gain valuable insights from, even when massive data sets are at play.
2. Collection & Analysis
With access to such granular intelligence, and user expectations at an all-time high, we’re dealing with millions of audiences of one now: individuals and real people rather than simple demographics. We need to fully understand who our audiences are, their needs and their emotional drivers as much as we can, to create better experiences on an individual level. And have those experiences ranked highly in SERPs, of course. The thing is, traditional keyword research may not be enough. According to IBM (2017):
88% of consumer conversations are now in private messaging apps.
Using data we have available across multiple channels, from Social, to email and even chatbots, we have the ability to draw insights around specific issues our customers have that go way beyond the remit of keyword research. Developing a more holistic approach to keyword research that incorporates the wider sphere of communication around a brand, it's possible to develop insights around brand sentiment, customers’ attitude to competitors, and even personality attributes which may impact their behaviour.
While we're probably not quite there as an industry yet, there are interesting moves being made using machine learning to speed up the analysis of large datasets. By using tools like Watson's natural language API, we can build a more detailed and rich view of the user that goes way beyond traditional keyword research and gets us closer to the audience's needs as a whole.
3. Hypothesis & Ideation
As Avinash Kaushik once rightfully said, “all data in aggregate is essentially crap”. It’s essential that we take all this analysis and create hypotheses which can then be applied to form ideas around making improvements to the user experience. We can use Google Analytics segments to inform hypotheses around behavioural traits and specific audience groups based on real data rather than assumptions.
Furthermore, this process can be applied in line with your engagement scale to identify key groups of users, their pain points and weaknesses in your content or their journey.
4. Testing & Iteration
Split-testing at scale for SEO is a commonplace tactic as part of an integrated SEO strategy. However, we can go further nowadays by drawing on what we might previously have thought to be more CRO-focused practices. There are options we can apply at a landing page level to test the hypotheses we formed. Platforms such as Google Optimize (I’ve written about getting started here) enable us to test specific iterations across key segments, measuring the performance of variations in terms of user experience and engagement.
Having tested, we can deploy the winning variation, or perhaps dig deeper and try again. We have at our disposal, platforms such as Sitecore, that enable us to serve personalised experiences to specific audiences permanently, creating those laser-targeted relevant experiences we’re all looking for as marketers (and consumers!) At this point, we rinse and repeat.
Combining SEO and CRO for a better overall Search experience
Machine learning is shaping a more fluid, intelligent, user-first search engine. This is having a huge impact on SEO, with user experience metrics being used by search engines to rank content. In my opinion, we're at the beginning of a new phrase of SEO in which success will be achieved SEO and CRO teams collaborate, applying the mindset and rigour of each other's disciplines as part of a shared framework. The 'sweet spot' will come when these practices converge and user expectations are met.
