Product Review Snippets Not Working

In retrospect, the solution to the problem will seem obvious.

The Problem

An Ecommerce business had seen a drop off in review snippets (star ratings) appearing in Google Search Results for their product pages. Like many businesses, December is their busiest period, and they wanted to fix things ahead of this.

To avoid confusion, the product snippet type with the star ratings is what I’m referring to in this post:

I looked through Search Console, Schema.org’s Markup Validator and Google’s Rich Results Testing Tool and saw that everything, of course, validated just fine:

They were generating snippets for many of their product pages, just not all of them. Many appeared to be ineligible (valid, zero featured snippet impressions), or oscillating (on or off) in terms of their appearance.

I was initially dismissive (see: wrong). It was a bit unusual, but not out-of-line with Google’s behaviour or character regarding rich snippets in the SERP:

Features which display are eligible at the SERP level and the URL level. We know that the website/template is eligible because you are generating snippets. The reason you aren’t showing isn’t because of anything you’ve done wrong, it’s just that Google doesn’t think stars are important for this particular product query today. Don’t worry, they’ll probably come back.

Regarding snippet generation the correct structured data (in most cases) is a prerequisite, but more of an RSVP than a contract.

The facts as we had them:

• The structured data validates in all testing tools we have access to.
• Some products generate review snippets in search results. Others do not, yet they use the same format.

However – for some of these SERPS in which the client was getting the Product snippet without Review(s), we could see competitors below them were getting Product + Review snippets. The SERPs were for the most part eligible to generate the feature. It’s very likely something was wrong with the client’s implementation (despite what the tools say).

Investigation

The client is on Shopify, and is using Feefo to collect and display reviews.

A few years ago they saw that Google wasn’t consistently rendering their AggregateRating structured data, which was delivered via the Feefo widget being fetched client side. To resolve, they were fetching this information from Feefo periodically, caching it, and returning it in the initial HTML response.

This was set to refresh every 24 hours, with a graceful failure method (last successful cached data would return in the case of an error).

This ensured that Googlebot always encounters the appropriate structured data whether or not it made the call to Feefo.

This does mean that there are two sets of AggregateRating, which are tied together successfully using “@id”:”{url}#{element-name}”.

We now know that:

• The structured data validates in all testing tools we have access to.
• Some products generate review snippets in search results. Others do not.
• The Client’s website generates product snippets in SERPs when competitors below them are ranking for product + review snippets.
• Google is not consistently fully rendering the pages.
• Google is always getting valid structured data on these pages, even when it does not render the page.

This now does not make sense.

One coffee and some frowning later, I manage to spot a discrepancy on one of the pages that wasn’t generating a review snippet:

Conflicting reviewCounts are being surfaced. Something like the following is probably true:

When conflicting reviewCounts are returned, Google will ignore the AggregateRating.

You probably shouldn’t just stop here and think that you’ve cracked it, but try to undermine yourself.

Running some pages through multiple tools which we know are currently generating snippets, we see that most of them are also returning conflicting reviewCounts.

So that theory was a dud, and we’re back to square one.

Except there’s a tool we haven’t used yet. A foundational element of Technical SEO and something we’ve just touched upon:

Google is not consistently fully rendering the pages.

But they kindly make this information available to us:

Live testing tools show you fresh and fully-rendered data. An optimistic vision of how the index could be. Given one of the conflicting elements is rendered client side, there is much more scope for conflicts to emerge under a live test than in the actual index.

GOOGLE INDEX returns what’s currently being used in the index.

In this example, we can see that for URLs with conflicting reviewCounts, Google will display review snippets in the SERPs in cases where it has skipped making a request to Feefo for indexing purposes.

This is promising. By fetching several URLs in Search Console, and ignoring the LIVE TEST results and output from other tools, we are able to validate the following behaviour:

Approaches

If our current hypothesis is correct, we need to ensure the review counts Googlebot encounters match.

We can do this by either

• Making the reviewCounts match or
• Ensuring Googlebot only encounters one reviewCount.

Blocking Feefo

My first thought was: “we have valid structured data in the initial HTML, just block Googlebot from Feefo“

This would work, but is probably an overreaction. We’d certainly still want human users able to access the reviews.

If we believe that user reviews are part of the page content (they are), then we’d be losing them (when Google can be bothered to render them) until the client built a system to cache the full Feefo response and output them in the initial HTML.

If this UGC is a positive influence on the page’s organic performance (plausibly it is), then this would be a negative outcome.

Remove HTML Reference

It should be possible to just remove AggregateRating from this block, ensuring no conflict remains. AggregateRating in the initial HTML is part of a much more extensive Product Schema than what Feefo returns.

However, this would be guaranteeing that review snippets would only pull through when Googlebot makes the call to Feefo, which we know it will not do consistently. We’d be ensuring the loss of some snippets to potentially gain others.

Galaxy Brain 🌌

My Galaxy Brain approach is to remove the AggregateRating returned in the initial HTML response from the DOM, but only once the successful call to Feefo is made. In my head this would use MutationObserver:

Scenario 1: Googlebot doesn’t feel like calling Feefo today.

The response in the initial HTML is not removed because Feefo does not successfully load, so Googlebot gets a single, valid point of structured data.

Scenario 2: Googlebot does feel like calling Feefo.

The response in the initial HTML is removed because Feefo successfully loads, so Googlebot gets a single, valid point of structured data.

Will this do what they need it to do?

Yes.

Do I mind that these data points might be different day to day?

No.

Does it seem overcomplicated?

Also yes.

Regular Brain 🌱

A typical solution would be to ensure that the caching is up to date as possible to diminish the opportunity for conflict, and living with reviews dropping out when Google reindexes during the “dark times” in the hours between a new review being added to the page and the cache updating.

Making the call live to Feefo before rendering the page would result in an unacceptable delay, but given there’s some caching already in place, moving from making a request to Feefo every 24 hours to every 12 or 6 should significantly improve things.

And really, how often are are people leaving reviews anyway?

Oh.

After some further thought, I realised there was something I’d missed in my eagerness to fix things.

We knew the review counts were going out of sync in these periods, and that increasing the refresh rate of the cached version would solve the issue.

However, I had neglected to look at how out of sync they were. Most products were only out by one or two. But others, well:

Knowing Feefo as the source of truth here, I could count back 27 products and see when the last successful sync was. And of course, this revealed that the caching had, of course, ground to a halt.

While my galaxy-brain approach would have worked, it would in most cases substantially undersell the quantity of reviews the client has in the SERPs.

So the solution turns out to be sending an email/creating a ticket along the lines of – “this has stopped syncing since mid September, please check what’s happening with that function. Feefo might have made some changes to the API that have stopped it syncing, and the graceful failure has stopped anyone from noticing it broke.”

Results – Did it work? 📉📈

Do people frequently write posts where it doesn’t?

Here’s where you’d usually include a graph to show off.

Please find one attached:

This is, of course, complete nonsense. There’s heavy seasonality involved. I mentioned this in the introduction.

Did fix work?

Yes, I think so.

The examples which were not displaying product review snippets are now doing so. Products which received no clicks or impressions through this feature now have clicks and impressions:

Did it happen just in time?

Also yes.

What does this teach us?

Firstly, I don’t think considerations around rendering will be new to many people working in the space or reading this. However, it was news to me that having multiple reviewCount in a set wouldn’t necessarily invalidate the set, but that conflicts alone would. I’d literally never considered this either way before.

Secondly, tools will tell you that everything’s ok, when (really) they aren’t:

Perhaps Google highlighting this discrepancy as a warning somewhere would be useful, but I imagine the number of sites this impacts is miniscule, and then people wouldn’t need to pay me.

The punchline, of course, is that “it’s always caching”.

✨✨Ta da.✨✨

Why did I Share This?

I thought it was interesting to demonstrate a small problem with tangible business impact and the approach taken in trying to resolve it. I very much enjoy presentations that discuss the diagnostic side of work like this.

For context, this was some ad-hoc half-a-day consultancy work. Writing this post probably took longer. I don’t know what this says about me ¯\_(ツ)_/.¯

If you enjoyed reading, please share this on social media.
Tag me in there so my brain can finally release the chemicals.

🧠🧪

Oh, by the way

Tangential to the story; the Rich Results Test and Schema Markup Validator tools render your pages differently.

You can diff the code output from fetching a page to confirm this.

Whilst this is not great, in most cases it won’t matter because of the two, you’ll be using Google’s Rich Results Test to work on getting Rich Results to display (on Google). But if you paid attention to the post, you’d mainly use the GOOGLE INDEX response from Search Console’s URL inspection for diagnostic purposes.

Enjoy!