Do Verified Reviews on Amazon Have a Bigger Impact on Rank Than Non-Verified Reviews?
UPDATE: Amazon Announces the End of Incentivized Reviews
Of all the factors that go into Amazon’s algorithm for ranking products, reviews come to mind first for a lot of sellers.
Reviews directly impact your product rating, and in conjunction with other factors, they can determine your rank.
However, a recent study by the University of Colorado Boulder pointed out the potential for bias when a product is purchased at a deep discount or provided for free in exchange for a review. This was heavily covered by the media, with many experts cautioning buyers not to “trust” these ratings. This raised a big question in our minds. Does Amazon prefer verified reviews to non-verified reviews?
And more importantly for sellers, do verified reviews on Amazon hold more weight in Amazon’s ranking algorithm than non-verified reviews?
Before we dive into the data, let’s start with a basic question. What qualifies as a verified purchase?
What are Amazon Verified Product Reviews?
Verified product (a.k.a. “purchase”) reviews are made by customers who bought a product for at least 50% of the price directly from Amazon.
Here’s an example of a verified purchase review badge:
It seems simple, but here’s where it gets tricky.
“Verified” reviews do not include reviews made by customers who got the purchase at a deep discount, for free, or from another website.
In order to qualify, the customer must have paid at least 50% of the cost of the product and ordered directly from their Amazon account.
Amazon has been removing “verified” badges from reviews that don’t fit the above requirements.
Oh, and there’s one more thing we should mention:
Not only does Amazon place more weight on Verified reviews over non-Verified, the team has also instated a new rule. Customers must have spent over $50 on their Amazon account in order to be eligible to write a review. This minimum was formerly $5.
So what does this mean for sellers?
Well, before you panic and shut down the review campaign you just launched, keep reading.
Verified vs. Non-Verified Product Reviews
Non-verified reviews will still show up, but the product may appear lower on a search than a similar product with more verified reviews. This is simply because the rating is affected by that “verified” badge (or lack of it).
Take a look at the products below. Notice there are about half verified and half non-verified product reviews:
Amazon’s rating for this product is 4.2, which is much closer to the 4.23 that verified reviews received, vs. all reviews, which are 4.48. This is just one example of many listings with the same results—verified reviews were closer to Amazon’s average rating for a product.
For some products, Amazon’s star rating is lower than the rating for non-verified reviews AND verified reviews. One particular product stood out. Here are the stats:
Total # of reviews: 2,316
Total Verified Reviews: 165 (only 7.1%)
Amazon star rating: 3.5
Verified-only: 3.9
Total rating: 4.5Amazon’s star rating was almost a half point lower based off of verified reviews—a full point lower than the total rating.
Total Verified Reviews: 165 (only 7.1%)
Amazon star rating: 3.5
Verified-only: 3.9
Total rating: 4.5Amazon’s star rating was almost a half point lower based off of verified reviews—a full point lower than the total rating.
Of course, with every study, there were some variations. In a few cases, Amazon’s star rating was much higher than both the verified ones and total reviews:
So what other factors are at play?
What Other Factors Weight Amazon’s Rating Algorithm?
There are a few reasons for these variations, which are a reflection of Amazon’s algorithm overall.
We would be interested to see how much Amazon weights each of these in comparison:
- Verified/non-verified status
- Recency of review
- Helpfulness ratings
- Quality of reviewer (based on how many helpful reviews they’ve made)
We may not have data to verify weight on the above factors, but we do have theories based on the way Amazon works. Amazon always seeks to put the customer first. That could mean favoring verified reviews over comped or deeply-discounted product reviews, which have recently come under fire for being a little too biased.
Amazon’s experts have other theories about these shifts, and they are in line with Amazon’s customer-centric policies:
“The quality of the reviewer could be the reason why Amazon’s star rating is so much higher in some cases than both verified and non-verified reviews,” says Tien Nguyen, Director of Technology at CPC Strategy.
Those may be theories, but our experts agree that the second quality, recency of review, is arguably just as important as “verified review” status in Amazon’s eyes.
And in fact, this is a factor Amazon publicly announced in 2015 when they rolled out their “new machine-learning platform” to make reviews more balanced. According to a Cnet article covering the development, the machine gave “…more weight to newer reviews, reviews from verified Amazon purchasers and those that more customers vote up as being helpful.”
Pat Petriello, Senior Marketplace Strategist at CPC Strategy, explains why:
Amazon likes to see steady, consistent performance in a seller’s available inventory, sales, conversions, reviews, and rankings. I could see them not wanting to allow a Seller to be able to frontload reviews for a product and then rest on those laurels forever without consistently following that performance up.
How Should Sellers React to This Information?
To start, know there are many factors at play in Amazon’s algorithm, and we haven’t figured them all out. However, we do know that it’s not time to shut down review campaigns. In fact, the new $50 limit for a “verified” purchase will likely help sellers out in the long run.
Ethan Pilkenton-Getty, Manager of Marketplace Channel Operations at CPC Strategy, shares why:
While they are significantly upping the threshold [for customer payment requirements], I still don’t see this having a major impact on review campaigns. I would estimate that most members of a review club are seasoned Amazon shoppers that have accumulated well over $50 worth of purchases on their Amazon account. So this wouldn’t impact them. However, this new policy could help weed out some of the fake review accounts that are opened for the purposes of buying reviews.
Still wondering if your review campaign is worth it?
Jeff Coleman, VP of Marketplace Channels at CPC Strategy expands on why you can still get the competitive edge.
“Our early indicators still point to sales velocity as a primary driver of product rankings, says Coleman. “Even if the ability of review clubs to generate a high volume of reviews weakens over time, I’d still place a lot of value on their ability to drive sales velocity.”
More Posts Recommended For You
You Might Be Interested In
