Companies frequently want to tout that their products are the most popular, reliable or recommended products in the marketplace. Given the increasing popularity of online ratings on electronic retailer sites, can a marketer rely on such ratings to claim that its product is the most recommended by consumers? A recent decision by an important self-regulatory appellate body cautions against such reliance.
Euro-Pro Operating, LLC advertised on television and the web that its Shark brand vacuum cleaners were “America’s Most Recommended Vacuum” or “Most Recommended Vacuum Online.” Euro-Pro’s competitor, Dyson, Inc. challenged the claims before the National Advertising Division (NAD), a self-regulatory body overseeing national advertising.
The basis of the superiority claim was not that Shark was the best-selling vacuum, nor was it based on an independent consumer survey of recent vacuum purchasers. Rather, Euro-Pro based its claim on its analysis of online consumer feedback at leading national retailer websites. Euro-Pro identified upright vacuum cleaners accounting for over 85 percent of U.S. sales and analyzed customer feedback reviews for those vacuums at 11 retailer websites: Amazon, Bed Bath & Beyond, Best Buy, Costco, Home Depot, Kohl’s, Lowes, Sam’s Club, Sears / Kmart, Target and Wal-Mart. Online reviews on the retailer sites, except for three (Amazon, Costco and Target) included a “yes/no” response question asking if the consumer recommended the product. In its analysis of the eight websites that included a yes/no recommendation question, Euro-Pro determined that Shark vacuums had the highest percentage for any manufacturer and Dyson had the second highest percentage, approximately five points less.
In evaluating the challenge, NAD found that the advertised claims reasonably conveyed a message that Shark is the most recommended vacuum brand among American consumers of vacuum cleaners. NAD also agreed with Dyson that Euro-Pro’s online data analysis offered as substantiation did not sufficiently represent American vacuum cleaner consumers and was unreliable. NAD recommended that Euro-Pro discontinue its most recommended claims. Euro-Pro appealed NAD’s decision to the National Advertising Review Board (NARB) which has appellate jurisdiction over NAD decisions.
The advertising message
NARB (like NAD) takes the position that it can place itself in the shoes of consumers and decide what messages are reasonably conveyed by challenged claims even in the absence of reliable consumer perception evidence. In addition, these bodies take a hard look at superiority claims, frequently recommending against such claims unless there is objective and reliable support for the claim. Here, NARB agreed with NAD that Euro-Pro’s advertising reasonably conveyed a message that Shark was the most recommended brand of vacuum among American vacuum cleaner consumers. NARB even found that a disclaimer, which appeared in at least some of the advertising that the claim was “[b]ased on percentage of consumer recommendations for upright vacuums on major national retailer websites” in a given month would not be understood by consumers as limiting the claim to its narrow context. Rather, NARB found that the disclaimer would likely be understood by consumers as explaining the methodology used to substantiate the claim. NARB also found that even if the disclaimer was understood as limiting the claim, it would still be unacceptable because it found that a disclaimer cannot contradict the main claim made in an advertisement.
NARB noted that “most recommended” claims are ordinarily supported by consumer surveys designed to test the opinions of a representative sample of the population identified by the claim. While NARB agreed with NAD that traditional consumer perception surveys are not the only way to substantiate a “most recommended” claim, support for such a claim must be based on reliable data that is representative of the population identified as supporting the claim. Thus, NARB reviewed Euro-Pro’s substantiation for both reliability and representativeness.
Reliance on online reviews
NARB rejected Euro-Pro’s reliance on consumer feedback reviews finding that the data was not sufficiently representative. There were several reasons for this conclusion. First, NARB was persuaded that 84 percent of vacuum sales are not purchased online, but rather at traditional brick and mortar stores. NARB also found that the data was too heavily weighted to Amazon.com, which accounted for only 2 percent of upright vacuum cleaner sales in the U.S., but 42 percent of Euro-Pro’s data.
NARB also noted other problems with Euro-Pro’s data, including evidence that consumers who purchase vacuums online tend to purchase more expensive vacuums and that three significant online retailers representing nearly half of all reviews considered by Euro-Pro were not included in its analysis of consumer recommendations because those websites did not have a yes/no question asking the consumer if he or she would recommend the product. Given these factors, NARB determined that Euro-Pro did not show that the data used to substantiate its “America’s Most Recommended” claim came from a representative sample of American vacuum consumers.
NARB also found a number of issues that impacted the reliability of Euro-Pro’s data including that the recommendation questions were phrased differently across the various retailer websites, that there was no “don’t know” option and that “yes” was always the first option displayed which raises the possibility of a selection bias in favor of “yes.” NARB also noted that retailer websites did not have consistent policies as to how long a consumer review was displayed on the website which means that many reviews may have been reviews for outdated models. Accordingly, NARB also agreed with NAD’s decision that Euro-Pro’s data was not sufficiently reliable to support a claim of “America’s Most Recommended” as to vacuum cleaners or upright vacuum cleaners.
Marketers are constantly seeking to exploit positive online feedback from consumers regarding their product. Like a testimonial, such organic feedback can provide a powerful advertising message. Although such feedback is frequently looked at by consumers in evaluating an online purchase decision, marketers need to be careful that any repurposing of the data is not overstated and otherwise meets with the reliability and representativeness standards applicable to traditional advertising.