subscribe: Posts | Comments      Facebook      Email Steve

The irrelevance of blogs that say the 100-point system is irrelevant

11 comments

 

Back in the 1990s I was supplementing my wine-writing income by doing a little healthcare reporting. As things turned out, I became known as something of an expert in the intersection of the U.S. healthcare industry–the nation’s biggest–and the emerging Internet. Everyone from pharmaceutical manufacturers to insurers, hospital administrators and individual doctors wanted to know what would happen when these two gigantic forces met, and how they could incorporate this new-fangled World Wide Web into their businesses. I didn’t have a clue, to tell you the truth, but I knew just enough more than most people (including my editors) to keep me gainfully employed and give my writing the semblance of expertise.

One memory stays with me of that period. Dot-com startups were as common in those days as the squirrels now gathering nuts in my neighborhood here in Oakland as Fall approaches. Here’s how it typically worked (and this is a fascinating and useful glimpse into the troubled, and troubling, nexus of marketing and reporting, as well). Someone starts a dot-com company; let’s say it purports to keep Internet-based communications between physicians secure. Somehow, that new business, which may not yet even exist (which would qualify it as “vaporware”), attracts the attention of one of the big investment banks. That bank would buy a piece of the company or otherwise associate itself with it. Then the bank would make its analysts available to journalists writing about that topic; the analysts would help the hapless reporters understand the finer points of whatever the topic was. If you’ve ever been a working reporter, you know how much we depend on the kindness of analysts whom we can quote with certain knowledge that the quotes are accurate, because after all, that analyst is an expert who works for an important investment bank.

Well, you see the obvious conflict of interest. The analyst talks up the new company, explaining how the product or service it provides is badly needed, and that this company (which he has analyzed in detail) is in a good position to succeed. The company is, according to the analyst, a sure thing. Meanwhile, the reporter–me–is typing all this down, to incorporate into the story. Next thing you know, a big, glossy healthcare magazine is running it, complete with the analyst’s spin (but now in my words) about how great this new startup company is. It’s a win-win-win for everybody: the startup’s owner, the investment bank, the analyst, me, and the publisher of the magazine who’s paying me.

Except for one little thing: in many cases, the analysts either lied or were entirely incorrect (for whatever reason) in their judgments. As we all know, the dot-com era crashed spectacularly in the early 2000s. A lot of “sure things” perished overnight; a lot of people lost everything. Many of the “sure thing” predictions turned out to be as premature as reports of Mark Twain’s demise. That horrible era taught me some important lessons I’ve carried through in my journalism ever since: Be skeptical of claims, even by so-called  “experts” who seem so self-assured. I developed a B.S. radar that to this day serves me well. That radar always asks these questions of anyone giving information or advice: Does this person have a hidden agenda? What does he or she have to gain (or lose)? Is there solid evidence of this person’s claims?

When the dot-com collapse finally happened, I’d been out of healthcare writing for a few years. But I was shocked that my reporting had had something to do with instilling a sense of trust in these startup companies–trust that, as it turned out, they didn’t deserve. People all over the country had read my articles and made decisions based upon facts provided to me, and by me to readers, that had turned out not to be true. I vowed never to let that happen again.

We come now, after this somewhat labored intro, to the subject of today’s post. Over the last number of years, we’ve had many reports of the Death of the 100 Point System. These have mainly come from the wine blogosphere. We can now see that these reports have been characterized, not by critical judgment or factual data, but by wishful (and even magical) thinking. Typical of the genre is this blog post from last week that incorporates the standard memes:

-       people, especially younger ones, don’t care about point scores

-       they would rather get a recco from a friend than from some famous [old] critic

-       wine criticism is subjective anyway, so giving it a number is crazy

-       the only way to judge a wine is to experience it yourself

The writer then offers examples from his own experience to “prove” the truth of each assertion.

Well, of course, each of these bullet points is true in its own way. But they’re no truer now than they were pre-social media. Only a tiny percentage of wine consumers ever cared about point scores. But in fact, more people today are influenced by them than ever. More and more big retailers (Beverages & More, Costco) are using point scores (by people like me) to market their wines, so that more and more consumers are exposed to them. And believe me, these retailers wouldn’t use point scores if they didn’t know a high score moves SKUs.

And younger people, of course, always have been resistant to the advice of elders. That’s what it means to be young: There’s nothing inherently different about kids in their twenties today than at any other point in history. They’ve always been more inclined to respect the opinions of their friends than of their elders. Social media hasn’t changed that. The point is that young people will someday be middle-aged people (that’s the way it goes, kids), and when they have a little more money in their pockets, they’ll do what people with disposable incomes have always done: seek the advice of experts when it comes to buying things, like autos, high tech devices and, yes, wine.

There’s simply no evidence that the 100-point system is endangered or irrelevant. In fact it’s at the height of its impact. Virtually every major critic in the world uses it (or some variation of it). The writer of the blog post I referenced understands this full well: in his concluding paragraph (which is where you always want to make an important point to leave with the reader), he writes: “Ultimately, the 100-point scale is here to stay.” That disclaimer, you’d think, would nullify everything he’d said up to that point. After all, you can’t be “here to stay” if you’re irrelevant, can you?

  1. The 100 point system is clearly not irrelevant, but it is getting increased competition in its function to influence consumer behavior. If available, social recommendations may have more influence on some consumers (millennials?), but of course these recommendations aren’t as easy to stick on a bottle as a Wine Enthusiast score or a Gold Medal, so Costco and Bevmo might not find them as useful until digital/social marketing gets a better integration with in store shoppers whether this is through QR codes or apps like Good Guide. Clearly there is power in the social recommendation and it will only become more influential as integration progresses, and this will likely be at the cost of the 100 point system.

  2. george kaplan says:

    100 points encourages the taster’s own subjective impressions more than, say, 3-5 puffs or a statistical model with a 20 point scale and true rankings.

  3. I am the Chief Judge for the Central Coast Wine competition in Paso Robles. In talking to winemakers they repeatedly told me that they get more of a boost in sales from 90+ point scores than a gold medal. As a result of their input we have now switched over to giving both medals and assigning numerical scores. (As have several other competitions.) So the use of the 100 point system is increasing.

    I just hope that consumers understand its limitations. There ia a false aura of scientific certainty

  4. Heather McAleer says:

    The 100 point system is, unfortunately, far from irrelevant. The problem is so many big box stores, grocery stores, liquor stores, and restaurants are making buying decisions based off scores of these critics. Many places love to brag about how they have more 90+ point wines than anyone else, they cover their shelves with signs talking about all the scores over 90 points. The problem is, there are a lot of good wines out there that many wine lovers will never get to experience because wines haven’t been lucky enough (or paid enough money) to receive a 90+ point score and therefore the somms or grocery stores won’t give those wines the time of day and put them in their stores or restaurants. I like how one person’s palate gets to determine what I may or may not have a chance to buy and try for myself. I can appreciate what the blogger was trying to say…that the scores some random person gives a wine, really makes no difference on whether or not you will enjoy a wine. It’s true. But the question of whether or not the scores are relevant? The simple answer is, to your own palate, it makes no difference. To the wineries trying to get their wines to us wine aficionados, it makes all the difference in the world!

  5. “There ia a false aura of scientific certainty”

    Is there really? How so?

    I think consumers are pretty used to the concept of subjective ratings scales. We get that there’s no scientific certainty behind why this movie or restaurant gets a “A-” or “3.5 stars” and that one gets a “B+” or “3.0 stars.”

    Is there something inherent about wine that makes consumers supposedly forget that concept when it comes to wine reviews? I find that hard to believe.

    Or is it the use of a 100-point scale as opposed to letter grades or “stars”? Are there really consumers who think that there’s some rigorous, unimpeachable, objective reason why a reviewer gave a wine 94 points instead of 93?

    I’m usually pretty cynical about the intelligence and sophistication of the average person, but I think people who look at rating scores know pretty much what they’re good for — a useful shortcut for whittling down a sometimes bewildering array of options.

  6. Jim -

    I have to side with Bob. His point is that people attach some kind of permanence to a wine’s score. They’ll say, “This is a Cabernet,” (which is true), and, “This is a 94-point wine” (which implies some kind of universality of the point score). I hear that stuff all the time, from friends who are much smarter than I am but don’t think all that much about wine. They like to buy it and drink it, and they like to look for wines based on point scores.

    Bob –

    Does this mean you’ll be publishing point scores of wines in your competition that do poorly? Or will you decline to tell the public which wines were not well received, as most competitions do?

  7. Evan, I don’t dispute that many people care about point scores and buy wines accordingly. As Steve noted in the post, store owners don’t post scores just for fun.

    I just don’t think that consumers are assuming some kind of scientific objectivity as Bob said. Saying “this is a 94-point wine” can just be a shorthand way of saying “someone who knows a lot more about wine than me said this was a very good wine.”

    I don’t see it as any different than looking at movie critics’ year-end Top Ten lists or Academy Award nominations. I don’t think anybody believes that there’s an objective way to measure movies, but if you’re wondering whether you missed any good movies this year that you might want to catch up on later, those lists can serve a purpose.

  8. doug wilder says:

    I still think the best way to deal with those who think the 100 point scale is irrelevant is to ignore them. I continue to use it because it is something my subscribers overwhelmingly tell me they value. I don’t doubt that friends pickup wine buying tips from each other, however there needs to be something that encourages the first bottle purchase of a particular wine out of thousands of choices. If a new release, they couldn’t have learned about it from a friend so what made them pull the trigger? Chances are it was either a Sommelier, Merchant, Educator, Journalist or Critic that helped guide them to that choice based on their preferences. These professions all have their specialized insights into wine and taken together there isn’t much in the world of wine they don’t know about before the consumer does.

    As critics, we taste wine for a living, making assessments on thousands of bottles each year, maybe finding a small fraction that really excel. Consumers look at wine very differently essentially wanting a bottle that delivers what they expect on a consistent basis. During my years in wine retail, I don’t recall once where a customer stopped to call a friend for a confirmation on a wine, whereas I wish I had a nickel for everytime I was asked “what did Parker give it?”.

  9. doug wilder says:

    I still think the best way to deal with those who think the 100 point scale is irrelevant is to ignore them. I continue to use it because it is something my subscribers overwhelmingly tell me they value. I don’t doubt that friends pickup wine buying tips from each other, however there needs to be something that encourages the first bottle purchase of a particular wine out of thousands of choices. If a new release, they couldn’t have learned about it from a friend so what made them pull the trigger? Chances are it was either a Sommelier, Merchant, Educator, Journalist or Critic that helped guide them to that choice based on their preferences. These professions all have their specialized insights into wine and taken together there isn’t much in the world of wine they don’t know about before the consumer does.

    As critics, we taste wine for a living, making assessments on thousands of bottles each year, maybe finding a small fraction that really excel. Consumers look at wine very differently essentially wanting a bottle that delivers what they expect on a consistent basis. During my years in wine retail, I don’t recall once where a customer stopped to call a friend for a confirmation on a wine, whereas I wish I had a nickel for every time I was asked “What did Parker give it?”.

  10. Sorry, Steve but this -

    “…young people will someday be middle-aged people (that’s the way it goes, kids), and when they have a little more money in their pockets, they’ll do what people with disposable incomes have always done: seek the advice of experts when it comes to buying things, like autos, high tech devices and, yes, wine.”

    - is almost certainly false on its face, especially when considered in the light of data on what wine consumers actually buy and do. See http://www.afr.com/p/national/friends_most_influential_in_wine_a8WuErUwMk6eM57VuGfFlN for an example citing Neilson data (summary: people see peer recommendations as most important when selecting a wine).

    We have **already** moved to a place where U.S. consumers purchase ALMOST EVERYTHING differently, factoring in multiple sources of reference. Children do not go back into the womb, genies do not go back into the bottle, cats do not go back into the bag, and consumers do not remain stagnant in how they approach (relatively complex) buying decisions. This is true regardless of the fact that the behavior does not conform to previously successful but now outdated business models (as the print and music worlds learned the hard way).

  11. STEVE,

    ON THE SUBJECT OF WINE MERCHANTS AND THEIR RELIANCE ON SELLING OFF 100 POINT SCALE SCORES (AND THE RISE OF “MILLENNIALS” AND SOCIAL MEDIA).

    ~~ BOB

    Excerpts from Slate
    (posted July 21, 2011):

    “The Greatest Wine Retailer in America;
    How Chambers Street Wines eschewed critic ratings and built a loyal following.”

    http://www.slate.com/articles/life/drink/2011/07/the_greatest_wine_retailer_in_america.html

    By Mike Steinberger
    “Drink: Wine, Beer and Other Potent Potables” Column

    Depending on your circumstances, visiting a great wine shop can be an exhilarating experience or it can leave you feeling like a eunuch at an orgy. I’m on an austerity plan these days, and while I can walk into most wine stores content merely to browse, there is one shop that I actively avoid in the interest of financial rectitude and domestic tranquility. That would be New York’s Chambers Street Wines. IF THERE’S A BETTER WINE PURVEYOR ANYWHERE, I HAVEN’T ENCOUNTERED IT; at Chambers Street, temptation lurks in literally every rack and bin, and even just writing about the place makes me want to whip out a credit card. . . . BUT IT IS ALSO THAT RAREST OF THINGS IN AMERICAN WINE RETAILING: A STORE WITH A DISTINCTIVE VOICE. [Capitalization added for emphasis. – Bob]

    . . .

    THERE IS ANOTHER THING THAT SETS CHAMBERS STREET APART FROM MOST OF ITS COMPETITORS: LILLIE AND WOLFF HAVE NEVER USED RATINGS FROM CRITICS TO HELP SELL THEIR WINES. WHEN CHAMBERS STREET OPENED, IT WAS DIFFICULT TO FIND AN UPSCALE WINE STORE THAT WASN’T COVERED IN SHELF TALKERS TOUTING SCORES FROM ROBERT PARKER AND THE WINE SPECTATOR. MANY MERCHANTS HAD SIMPLY STOPPED SELLING WINE AND WERE INSTEAD FLOGGING POINTS. BUT LILLIE AND WOLFF WERE INTENT ON ESTABLISHING A RAPPORT WITH CUSTOMERS THAT WASN’T MEDIATED BY THIRD-PARTY OPINIONS. “We wanted the shop to be completely personal — to get know people’s taste, and to recommend wines we liked and that we thought they would enjoy,” Lillie told me. He’s quick to note that, back in 2001, the kind of wines that he and Wolff were interested in didn’t get much attention from critics, which made it easier to eschew scores. Fair enough, but I still think it took some guts make Chambers Street a points-free zone.

    A DECADE ON, THEIR DECISION LOOKS PRESCIENT. THAT’S BECAUSE RATING SEEM TO BE DIMINISHING IN IMPORTANCE. A VERY SELF-CONFIDENT WINE CULTURE HAS TAKEN ROOT IN THE UNITED STATES: PEOPLE ARE USING DISCUSSION BOARDS AND SOCIAL MEDIA TO FIND THEIR WAY TO GOOD BOTTLES, AND THE INFLUENCE OF CRITICS IS WANING, ESPECIALLY AMONG YOUNG DRINKERS. … RAMPANT GRADE INFLATION COULD BE HASTENING THAT DECLINE. High ratings help merchants sell wines, and being cited on shelf talkers and in email offers is free publicity for critics, who thus have an incentive to bump up their scores. But big numbers have now become so prevalent that they’ve turned the 100-point scale into a farce. I THINK RETAILERS ARE GOING TO HAVE TO LEARN TO SELL WINE AGAIN, and in that sense, Chambers Street has a big jump on a lot of other stores.

    . . .

Leave a Reply

*

Recent Comments

Recent Posts

Categories

Archives