Consumer Reports released its annual reliability survey on Thursday, but the findings are being overshadowed by a growing rift between the publication and Tesla, which took serious issue with the reliability score given to the automaker’s Model 3. Tesla called CR’s coverage “consistently inaccurate,” but CR responded on Friday, saying Tesla’s statements contained “supercharged and unsupported claims about the survey.”
“Tesla seems to misunderstand or is conflating some of what we fundamentally do—our Annual Reliability Survey report and the related predictions versus our car reviews and tests,” CR said in a lengthy response Friday.
A Tesla spokesperson deferred to the statement issued yesterday, when the automaker asserted the publication’s coverage has been “misleading to consumers.”
“Regarding its predicted reliability rating for Model 3, it’s important to note that Consumer Reports has not yet driven a Model 3, let alone do they know anything substantial about how the Model 3 was designed and engineered,” Tesla said.
The publication agreed with one of Tesla’s complaints—that it’s the subject of outsized coverage compared to other cars. That shouldn’t be a surprise, though, CR says. The Model 3 is portrayed as an all-electric, more-affordable sedan that could seriously upend the industry, and as a result, “Tesla does garner an outsized level of attention, scrutiny and discussion by the media,” CR wrote.
Here’s the first main point:
First, Tesla appears unhappy that CR expects the new-to-market Tesla Model 3 to be of average reliability, which is generally a positive projection for any first model year of a car. This expectation is based on CR’s 2017 Annual Reliability Survey, measuring the dependability as opposed to the satisfaction, of more than 300 car models, model year 2000 to 2017, using the responses of individual owners of more than 640,000 vehicles. We provide this information to help people make informed purchasing decisions as new products reach the market.
The publication says it generates data through surveys taken of car owners, which allows it to make predictions of cars it hasn’t seen in person. Jalopnik reported Thursday that CR’s practice of making prediction ratings for cars they haven’t driven began a few years ago.
In particular for the Model 3, CR said it reviewed more than 2,000 consumer survey responses Tesla models—which also led to the Model S receiving an “above average” reliability for the first time. The Model S is now CR’s top-rated car, the publication says. “Kudos on both, Tesla!” CR says, in what’s surely a jab at the automaker’s long, pointed statement this week about the publication.
CR goes on to address Tesla’s chief issue with how it produces ratings:
CR conducts a battery of 50 standardized tests across all the vehicles we review – we have a lot of mileage in this arena. We also continuously update our ratings as new surveys are conducted and we test the cars we purchase to reflect the current realities of what a consumer should expect in the marketplace. (That’s right, purchase. CR does not accept any advertising and purchases the products we rate like any other regular person.) The Model S rating has changed over time, going up and down, as new data becomes available.
That independent streak is part of the reason why CR’s take on cars is held up with high regard in the industry. Like the Model S, CR said, it plans to continue reporting on the Model 3.
“As with all the cars we review, you can rest assured that we will thoroughly test and evaluate the Model 3 with the same care and scrutiny we apply to all the cars we test just as soon as we can get one—we’re waiting patiently along with other consumers,” CR said.
Asked if it has any indication when the publication will receive a Model 3, a spokesperson told Jalopnik it’s not offering further comments at this time, though an editor said yesterday that the publication was on Tesla’s waiting list.
Tesla has said that around 455,000 orders have been placed for the Model 3, though the company has yet to fill the vast majority of those orders.
Here’s CR’s full statement below.
Late yesterday, Tesla shared with select journalists what appears to have been a prepared statement of supercharged and unsupported claims about the performance and safety of their own vehicles and our 2017 Annual Reliability Survey results, taking the occasion to air a number of grievances against Consumer Reports (CR) and our overall reporting on Tesla and its products.
As is often the result of any new product or company that electrifies the market, Tesla does garner an outsized level of attention, scrutiny and discussion by the media. While we appreciate Tesla’s efforts to typically embrace and navigate, if not directly steer, this attention, we would like to offer some clarity on the examples they cite. (For other, perhaps not surprisingly Tesla-positive, examples from CR, you can visit the articles currently available at the Tesla press site, at least until they pull those links down, or visit us at CR.org).
Tesla seems to misunderstand or is conflating some of what we fundamentally do — our Annual Reliability Survey report and the related predictions versus our car reviews and tests.
First, Tesla appears unhappy that CR expects the new-to-market Tesla Model 3 to be of average reliability, which is generally a positive projection for any first model year of a car. This expectation is based on CR’s 2017 Annual Reliability Survey, measuring the dependability as opposed to the satisfaction, of more than 300 car models, model year 2000 to 2017, using the responses of individual owners of more than 640,000 vehicles. We provide this information to help people make informed purchasing decisions as new products reach the market.
Here’s how we make the prediction: CR uses survey data it receives from car owners to predict the expected reliability of new cars being introduced to the market by looking across a manufacturer’s historic results (akin to how a weather forecaster predicts it will be sunny) — separate from the hands-on road tests we use for our overall score.
For the Model 3, we looked at more than 2,000 consumer survey responses about Tesla models. In fact, the Tesla Model S is now reported as having above average reliability for the first time ever. The Tesla Model S is also currently CR’s top rated car, period. (Kudos on both, Tesla!)
Second, Tesla has taken larger issue with how CR produces car ratings, citing specific examples where they think our testing methods fell short or were unfair. CR conducts a battery of 50 standardized tests across all the vehicles we review — we have a lot of mileage in this arena. We also continuously update our ratings as new surveys are conducted and we test the cars we purchase to reflect the current realities of what a consumer should expect in the marketplace. (That’s right, purchase. CR does not accept any advertising and purchases the products we rate like any other regular person.) The Model S rating has changed over time, going up and down, as new data becomes available.
Thanks to technological advances such as product changes delivered by an over-the-air software update and thereby adding or subtracting features, we reevaluate products to inform consumers about what to expect after any update. These changes are then reflected in our ratings. Tesla frequently updates its software in just this way, which is relatively unique in the automotive market, often resulting in material changes to its products and therefore our ratings — both positively and negatively. It also happens to drive more frequent press coverage given the need to communicate product changes to consumers.
While our reliability survey data feeds into the overall score we give any product,that is just one input. As with all the cars we review, you can rest assured that we will thoroughly test and evaluate the Model 3 with the same care and scrutiny we apply to all the cars we test just as soon as we can get one — we’re waiting patiently along with other consumers.
As an independent, nonprofit organization that works side-by-side with consumers to create a fairer, safer, and healthier world, CR provides trusted knowledge people depend on to make better, more informed choices. We conduct evidence-based product testing and ratings, rigorous research, hard-hitting investigative journalism, public education, and steadfast advocacy on behalf of consumers’ interests. Buying a car that has an average or above average score for predicted reliability will likely reduce the chances of having problems with the car.
We at CR are confident in our data, methods, and reporting — and the historic results we’ve achieved in improving consumer products, services, and the marketplace. We will continue to report on and test Tesla’s products in the same fair-minded, consumer-focused way we do with all manufacturers, to help shape products to best serve the needs of consumers.