subscribe: Posts | Comments      Facebook      Email Steve

The 100 point phenomenon and score inflation

16 comments

Eighteen 100 point Bordeaux wine scores in a single issue of The Wine Advocate? That’s what Andrew Jefford reports on his blog on Decanter’s website. When I read that, I immediately flashed back to my conversation with Allen Meadows, the Burghound, who told me that he’s given 100 points to only one wine in all his years of reviewing (1945 Romanée-Conti, which already was very old when Allen tasted it).

There is a theory out there, called score inflation, which borrowed its name from “grade inflation”, which Wikipedia defines as “the tendency of academic grades for work of comparable quality to increase over time.” I don’t think anyone would mind if academic grades were increasing in America because kids are actually getting smarter, but sadly, that’s not the case. The reasons why grades are on the uptick (if they are) has to do with political and social issues I don’t pretend to understand, but I do understand that grade inflation troubles many people, who perceive it as somehow bad and indicative of deeper problems in the society.

This aspect of being troubling also accompanies wine score inflation. Dr. Vino touched on it the other day, and while he didn’t exactly condemn it, he did kind of cast a mild aspersion on it, especially in his opening question: “Is inflation crippling wine scores?” The answer seems to be inherent in the question: Yes.

Okay, kids, let’s talk about score inflation. Now, right upfront I don’t think you can accuse me of score inflation, because I’m almost as stingy as Allen Meadows about 100 point scores. Ninety-nines and 98s too, and even 97s are hard to come by if I’m doing the rating. I don’t feel any pressure to give more high scores–well, maybe a little pressure, here and there–but in general, I’m happy with my scores, my employer is happy with my scores, and the only people unhappy with my scores are wineries that get low ones.

I do think that being too generous with high scores isn’t a good thing, though. To me, a 100 point wine should be one of the most uncommon things in life, like a bar mitzvah, turning 21 or having sex for the first time (which makes me think of Oscar Levant‘s crack about Doris Day: “I knew her before she was a virgin.”). Well, all right, you can only have those things once in a lifetime, whereas you can have multiple perfect wines. But it makes me wonder, where’s the line when it comes to 100 points?

To me, 18 in a single issue is too much. Granted, I haven’t tasted those wines, so maybe Parker really did find 18 instances of sheer perfection (and, as Jefford reminds us, “And another eleven with 99 or 99+ points”). But when so many wines are “perfect,” then it’s the Moynahan effect of “defining deviancy down” applied to wine. Call it “defining perfection down.” Another way of looking at it is Garrison Keillor’s closing shtick from his Lake Woebegon routine on A Prairie Home Companion: “…all the children are above average.” Obviously, all the children cannot be “above average” because then there wouldn’t be an “average” for anyone to be above or below.

I don’t think there’s anything particularly ominous about score inflation. It doesn’t harm anyone, and it makes the proprietors who get the perfect scores happy. But to me, to call a wine “perfect” (which is what 100 points means) must necessarily be rare. I’m glad I’m not as stingy as Allen Meadows, but I’m also glad I’m not as promiscuous with high scores as the folks at The Wine Advocate.

  1. James McCann says:

    In what other industry do their critics criticize each other??

  2. I’m sure Siskel and Ebert got into it with differing opinions on a few films, but I don’t know if they ever published articles on each other. I do know that food critics love to pan each other for praising a restaurant they feel is all hype or shock value. Take Anthony Bourdain for example; while he doesn’t give scores for Zagat or anything, he is certainly considered a critic and has published opinions of other critics in the field.

  3. The idea of score inflation is very evident. When 18 perfect scores are given in a single vintage for a single region it completely takes away the magnitude of the wines and their scores. The whole reason to have a grading system is to be allowed to show differences in quality representative by a numerical value. It really does not seem like a 100 point system anymore, more like a 10 point system where anything less than ten seems negative to all who care. I think the Burghound is the one of the last who is effectively using the 100 point system correctly.

  4. I would hate to be the winemaker who received only 98pts in that issue. He/she is probably looking for another job!

  5. Mr. McCann–

    The misuse of the rating system that so many of us employ hurts us all. Either we raise our scores to keep ourselves on some kind of equal footing or we risk being marginalized because high ratings, like it or not, draw attention to themselves and away from lower ratings.

    Steve has criticized Parker, but what he has really done is raise the issue of grade inflation. I still like numerical ratings, but if we all followed the Parker model, there would be damn few scores below 90 points.

    In addition to all those 100-pointers, he also gave out 99+, 99 and 98+. Unless someone cares to argue that all fifty or so of those wines are somehow better at 100 than at 98+, it is clear that the only reasonable interpretation left is that all fifty of them are incredible and virtually indistinguishable qualitatively.

    While I believe that Parker’s ratings reflect his enthusiasm for the vintage, the manner in which he has reflected that enthusiasm has hurt the consumer who would wish to follow his guidance. Put frankly, there is no guidance.

    Steve and I make our livings in the same industry as Mr. Parker and all the other critics. When one of them does things that are antithetical to good practice, it is not out of bounds to say so.

    It is no different from the San Francisco Chronicle’s critical comments about the Los Angeles Times decision to no longer attach star ratings to its restaurant reviews. In fact, Steve’s comments, and mine over on my blog, are at least about a perceived abuse, or an exercise in hubris, whereas Michael Bauer has simply said that the LA Times is wrong.

  6. I’m not sure I agree with the score inflation hypothesis. Is it possible that wine is just simply getting better? There are more formal viticultural programs turning out more educated winemakers, while technology has improved everything from vineyard layout to environmental consistency during fermentation and barreling.

    I think I would be surprised if wine hadn’t, on the whole, improved significantly over the past 20-30 years. So maybe we’re just seeing a point where more wines are able to achieve perfection.

    Would it be better if we moved the goalposts, so a wine that would have scored 100 points 20 years ago rates a mere 97 today?

  7. So the guys who got 99 (or 99+, so near, yet so far away), are they scratching their heads,wondering, “Where did I go wrong?”

  8. Brooks asks a good question, and I think largely the answer is yes, the chances of making wines of high quality has increased based on replanting, technology, education, and strenuous demands vintners place on the quality of the fruit grown, harvested and selected.

    Any winemaker will tell you that the quality of the fruit is paramount to being able to make exceptional wines. All of the other factors I mentioned(and I am sure I left out a dozen or more) all point towards attaining a consistent, high level of quality. I don’t taste nearly as prolifically as Steve or Charlie but I know that we all are faced from time to time with wines that may flirt with the ultimate accolade, which by definitition should be special experiences. For me, I can count on one hand the producers I have given perfect scores to in the last 8 years. That is out of nearly 10000 wines, all tasted non-blind. We all occasionally are responsible for an outlier, either high or low. It doesn’t make any of us irresponsible. Has anyone considered that Bob Parker can now concentrate completely on Bordeaux, unfettered by California, allowing him the time to dig even deeper? Personally, I haven’t even looked at the BDX scores in issue 199 to understand what the fuss is about.

  9. James McCann says:

    Mr. Olken,

    If Robert Parker’s customers buy Bordeaux based on his (possibly) inflated ratings, then he will lose their respect and business.

    I just don’t follow the logic of why other critics worry about score inflation, as all of you put your reputations on the line each time you review a wine, and the market determines who is trusted.

    BTW, I find Steve’s reviews to be some of the most consistent and trustworthy in the business.

  10. @Alfonso, that’s exactly what I was thinking!

  11. Steve,

    Very surprised by the comment “I don’t think there’s anything particularly ominous about score inflation. It doesn’t harm anyone, and it makes the proprietors who get the perfect scores happy.” As Charlie mentioned above, I think misuse of the rating system (in this case grade inflation) hurts all critics, and especially those engaged in it, for all the reasons stated here and in the comments to Dr. Vino’s post.

    The market does determine who is trusted, and I would be interested to hear how you think Wine Enthusiast, or you personally, stacks up against the other major publications/critics. How would you know if your ratings are becoming less trusted due to inconsistency, inflation or some other reason until it is too late? Do subscription rates, advertising rates, or the ability of your scores to move product provide insight into whether or not your scores are becoming more or less relevant?

    cheers,
    Jim

  12. Loathe to chime in here but 18 instances of perfection in a single review? To me, in and of itself, each version of perfection should redefine the very word. Similar to the bell curve does in a classroom setting I’m inclined to believe there should be an even distribution. The new definition of a 100 point wine needs to reflect (to Brooks and Doug’s comments) the reality that chances of making wines of high quality has increased and remain a lofty, virgin like, status…

  13. First, score inflation is pretty hard to prove, so that leaves us mostly with speculation. Some one posted (I can’t remember where) on Parker’s previous “vintages of the century” and he’s produced fairly similar # of 100pt wines. Even if we score his palate against anothers’, it’d be under the guise that neither has changed except for one moving relative to the other. This is the issue with rating things that are ultimately subjective. Not that I have a problem with scores at all. The problem is that everything with Parker is inflated. His notes a full of “possibly the best wine from the estate, blah, blah, blah.” There isn’t any restraint.

    Personally, I think Parker’s score have probably always been inflated. Whether or not they’ve gone up over time is another story. Regardless he is an untrustworthy critic and I think he hurts the wine industry as a whole.

  14. Dear Jim Finley, I’m sure that there are people at Wine Enthusiast who can answer your questions about subscriptions and advertising. I, myself, don’t concern myself with those things. I just work hard and try to do the best job I can.

  15. Thanks Steve. I, for one, certainly appreciate your work and think the system is very useful to wine drinkers. I was just curious if, and how, wine critics would be able to determine if they were losing relevancy with wine drinkers.

  16. Jim Finley, thanks. I don’t know how to determine if I’m losing or gaining relevancy. I do my thing and the market decides.

Leave a Reply

*

Recent Comments

Recent Posts

Categories

Archives