Consumer Reports is lying to you

This week, I’d like to take a question about Consumer Reports (God, even the name is boring). Did you hear, BMW is the best?

Not so fast. Email me!

Question

Hi Mike –

I’m sure you saw the news that Consumer Reports ranked BMW the best overall car maker for 2024.

I know you’re a big Bimmer guy, and I often reference the article you wrote about BMW reliability. Do you agree with with their findings? They seem to be at odds with finding the soul of a car.

– Phil

Answer

Phil is referencing this article. I believe that CR and myself have come to similar conclusions through different methods. But at the end of the day, remember they have an agenda, and they need to bring in money. Membership is required.

Do I have an agenda? Not really – those ad banners you see pop up on this site regardless of my words. So let’s take a look at how CR scores cars, and we can debunk some of their vaunted methods.

Consumer Reports Road Tests

BMW M2 — I’ll bet you a dollar that the road test for a 2 Series says “tight back seat”.

For the road-test score, we put vehicles through more than 50 tests.

I’ve no doubt they do, but this is the poster child case for test numbers. First, the number 50 is simply a psychologically satisfying number. If they said “34 tests” – feels imprecise doesn’t it?

When they do this, they take rulers and stopwatches and go to town. That’s not how you test a car – as I’ve said, does it matter if one gets to 60 a tenth faster, or has half an inch more leg room in the back? Sit in it, drive it. Then drive another car. I promise, you’ll like one over the other and it has nothing to do with 50 different tests.

To be fair, maybe a Honda Odyssey is a tiny bit smaller in the back than a Toyota Sienna and you really need that space to prevent your kids from staging a cage match on your daily commute. But by whatever metric you measure a car, you’ll still like one better as soon as you sit inside it.

Predicted reliability

Range Rover — This car may break on you. Shocking, I know.

Predicted reliability ratings are based on problems reported by members from 20 possible trouble areas in CR’s Annual Auto Surveys.

Oh, this is my favorite – the idea that you can predict whether or not a car will break on you. That’s bullshit.

Sometimes a car has a design flaw, and more often than not a recall is issued to fix that specific issue. But the idea that CR can pinpoint an issue before it happens is silly. Case in point:

Both Porsche and Cadillac moved up 10 positions this year due to their improved reliability.

How many models did Porsche and Caddy change from 2023 to 2024? Zero. They are identical, at least in terms of how they might predict reliability. Porsche improved 10 places by not changing anything?

Or how about this article from November of 2023? Porsche ranks ahead of BMW in reliability, then it changes four months later?

Owner satisfaction

Porsche Macan — Porsche ranked third, down two points to BMW. I promise, their cars are often better.

Owner satisfaction predictions are based on whether CR members said in our surveys that they’d buy the same vehicle again if given the chance.

You purchase a car, and for the most part, it’s integrated into your life for three years. Like marrying someone, by the end of your time together you’re gonna know each other’s flaws. The seat bolster makes it hard to get in, while you unleash a deadly fart in said seat on the way home from work. It’s a symbiotic relationship.

But let’s say you have bad luck and get five flat tires in those 36 months. Or you have a bad dealer experience. Maybe the seat is uncomfortable for you because you gained 20 pounds – these events are not the car’s fault.

But Consumer Reports says it is. You wouldn’t get the same car again, and that means it’s bad.

Safety

BMW M3 — All cars are safe now. Just not safely driven.

Safety includes an assessment of any available crash-test results and extra points assigned to vehicles that come standard with key crash-prevention systems.

Every modern car, especially luxury ones, have excellent safety systems, and you can look at the NHTSA’s site for crash test results for free. A 3 Series gets five stars. A Lexus IS gets five stars (aside from frontal) – there, I just did the work for you, no Consumer Reports magic needed.

Green choices

Hummer EV — Am I green enough for you yet?

Green Choices reflect how many of a brand’s CR-tested vehicles earned the designation for being in the top 20 percent of vehicles with the cleanest emissions, according to the EPA.

Both the EPA, and thus CR, don’t take emissions from vehicle production (or recycling) into account, so this is again a made up category.

And no offense to BMW or Porsche, but I don’t see them producing a competitor for a Ford F-250. People need those trucks to work for a living, and they better be able to tow. 600 lb-ft of torque means more emissions no matter how you get there.

Don’t cry for Ford, which makes a ton of profit off pickup trucks, but please keep it all in perspective.

Who’s calling in?

Subaru placed second. Name one you want aside from the WRX.

My last point here is who’s responding to those surveys? Like an online rating, you’re only going to review something if you’ve had a negative experience or a really positive one. That means the data they collect is skewed one way or the other.

But BMW is still better

bmw-m8 — Would you not buy this M8 if CR ranked BMW last?

Take the lineup as a whole, and I’d still say that BMW really is one of the better makers today. Every car is comfortable, and the cockpits are logically laid out. iDrive might be a digital mess, but I’ve see plenty worse. They make the best engines too.

But it’s not because an M3 is a tenth faster, or because Aunt Phillis said her X2 with 24,000 miles has no issues.

However, I’ll also say this. There’s no reason you won’t be satisfied with a Range Rover or Jaguar, cars that bottomed out in Consumer Reports rankings. Hell, a Range Rover uses a BMW engine, and they are waaay nicer inside. Yes, you might need to bring it in for service more, but you’re going to get rid of it in three years for a new one anyway.

So Phil, yes I do agree. But soul will always matter more than those little red and green dots.

Want MWS to review your car?

If you live in the tri-state area and want me to check it out, send me an email!

Support Machines With Souls

Commissions may be received for product links on this site. Help out if you can.

I use Nikon camera bodies and lenses, a Westcott Ice Light 2, Manfrotto tripod, B + W filters and an iMac Pro to make the art you see here.