subscribe: Posts | Comments      Facebook      Email Steve

What makes one wine “better” than another? The score as metaphor


I somehow found myself once again in the crosshairs over at Dr. Vino’s blog the other day (and what a great job Tyler Colman is doing there). Tyler was writing about “wine score inflation” (his term, not mine, and I’m not sure those words even refer to anything in the real world). Citing the writings of others, Tyler suggested that scores from the best known critics have been on the upswing, a fact (if true) he finds worthy of investigation. He didn’t exactly accuse the critics of anything nefarious, but the smell of “something’s rotten in Denmark” wafted over his post, as if from a nearby swamp.

I wrote in: “…if scores are rising for certain categories of wine, it’s because quality is improving. Critics are simply perceiving that increased quality, and rewarding it with higher scores.”

That seemed pretty innocuous to me, a statement so logical on its face, no one would even bother to dispute it.

But, wham! It hit the fan. The insults, I’m used to, especially from the usual tedious suspects. What did interest me, though, were some more thoughtful remarks that raised interesting questions. For example, Keith Levenberg (I don’t know who he is) wrote: “Steve Heimoff’s claim that ‘if scores are rising for certain categories of wine, it’s because quality is improving’ and ‘[c]ritics are simply perceiving that increased quality’ has also been Parker’s refrain for years, and it’s a complete fallacy.” Keith bases this statement on an assumption that seems highly questionable: that “critics are the worst-situated of any of us to make the determination whether quality is in fact improving, because the wines they are tasting are deliberately made to elicit their approval.”

I replied: “Are a thousand wineries in California deliberately ‘tinkering’ with their wines to match my ‘personal preferences’? I think not. That’s real conspiracy theory stuff. Instead, wineries are crafting their wines to what they perceive is a genuine shift in the consumers’ palate, of which I’m just one little part.”

The claim that winemakers are deliberately appealing to certain critics’ palates has been around for a long time. I suppose it’s true, in a way, but what’s so strange about that? A winery is a business, just like the movies or automobiles. Spielberg makes the kinds of films he believes Americans want to see. Detroit makes the kinds of cars they think Americans want to drive. If a winemaker decides to go counter to prevailing consumer preferences, chances are he’ll go bankrupt. The reason I have an impact in the sale of wine is because I reflect the general consumer preference in America. I don’t manufacture it and I don’t lead it. I like these wines because they’re good, made in a style to appeal to the wine-loving American, which includes me. When they succeed, they deserve the high scores I give them.

Then Daniel wrote (I don’t know him either): “Steve, Is every wine that you have reviewed 95 points better than every wine that you have reviewed 94 points?” I answered: “Yes.” Short and simple. Daniel replied: “So, a 97 point, $50 Calif PN, is a better wine than a $300 95 point Napa Cab?” Again, I replied: “Yes.” But,  slightly troubled by something, I added, “’better’ is a complicated concept. I may blog on this soon, to better understand it myself.”

Is a 97 point $50 California Pinot Noir “better” than a $300 95 point Napa Cab? To begin with, let’s forget about the prices. They’re irrelevant. In what respect is a 97 point wine (at any price) “better” than a 95 point wine  (at any price)? After all, it earned two points higher; it had to do something for those extra points, no?

This is a great question, raising profound issues that, frankly, haven’t been thoroughly explored by any critic I know of, including me (which is why I said to “better understand it myself”). For some reason, a lot more people these days care about these things than used to be the case–which may be due to an Internet generation coming of age that demands the utmost transparency and explanation. So I thank Daniel for asking this question. In tomorrow’s post, I’ll attempt to answer it, and if readers seem interested enough, perhaps the conversation will last through the week.

  1. Steve,

    Here is another question as opposed to a possible answer to your question. When you review wines, are the scores a relative score to the broad peer group (whether defined either by variety, geography, and/or vintage, etc.) or is it a score in relation to every wine and variety available in the world?

    When I qualify wines with rankings, I inadvertently default to context in their broad group rather than how it stands among every wine in the world, and only then apply some thinking about how that group of wines…white or red, variety, 50 years old or 2 years old, rank in the context of all other peer groups. Certainly not a rigid science, but that’s what I come up with thinking about your interesting question. Curious how you and others think about this?

  2. Steve – If Keith Levenberg dropped his career to become a full-time wine writer, he’d be among the two or three best wine writers currently working. His prose is effortlessly clever and thoughtful. It’d be a good idea to read his semi-regular stuff:

  3. You are heading into some interesting territory. I can’t help but contextualize wines when tasting critically. For me, a wine that costs $9.00 and is clearly ahead of the $9.00 competition ranks higher than a $300.00 dollar wine that is very well made, better by far than the $9.00 offering, but falls short vs. its peers. What an unholy mess I would make if I tried assigning points. Good luck with this. I’ll be watching with interest.

  4. Ben, I think you’ve got it right with respect to peer groups. I’ve often written that it’s impossible to judge a wine in and of itself. It can only be done in the context of a vertical tasting.

  5. It is clear from the same old tedious discussions of this question that the only thing determining a score is how one person enjoyed it (or didn’t) at one moment in time.

    This question would be answered much more easily if you had pre-set criteria for awarding each point on the merits of aroma, flavor, texture, length and possibly age worthiness and food compatibility.

    Of course, to do that, one has to be able to accurately and reliably identify these sensory components of a wine….

  6. Steve,
    I would be interested in seeing you riff more on what it takes for a wine to land in the 95+ category. There is at least an impression that it takes a score of this level to significantly impact sales. Perhaps as evidence of this is a story told to me by another Cabernet producer who got a (I think it was) 92 in the WA, which unintentionally published his home phone number rather than the business number, thereby setting up a situation where it would be obvious which calls were generated by that score. But there were no calls. This led him to conclude scores in the low 90’s had no impact.

    Sometime back you said something like (and I have to paraphrase from memory here as I can’t find it in the archives) for a wine to be 95+ points it had to have a stellar strength, and you made a comparison to in the world of fashion, that there are clothes worn by models that are instantly recognizable for their quality, though you wouldn’t necessarily be pleased to see them worn by a friend or relative. So can a wine described with words like finesse, elegance, or balance make it to the upper end of the scale? Or is there an edginess required, like the models in Vogue?

  7. Steve,

    The issue of what makes one wine better than another is one of my absolute favorites. It is of course a very deep philosophical question that goes to aesthetics, perception, the nature of value and even issues surrounding social acceptance.

    If nothing else an investigation of the idea of “better” and of the nature of quality will probably yield some very exciting and intellectually satisfying insights.

    By the way, anyone who tastes and evaluates wines double blind is implicitly arguing that “value” plays no role in the issue of quality. I only note this because it begs the question, does relative value play a role in assessing the quality of a wine?

  8. Metaphors and analogies are what we use when we can’t articulate an idea.

  9. With due respect Steve, if you truly reflected the general consumer preference in America wouldn’t you be giving higher scores to low cost sweet red wines & moscato’s? 😉

  10. Mark:

    Is “consumer preference” and quality synonymous?

  11. Tom
    If you buy into the idea of wisdom of the masses, and if you are buying what Robin Goldstein is selling, then the answer is a resounding “YES”.

  12. Better is in the eye (or nose) of the beholder. Just because Steve likes the $50 Pinot more than the $300 Cab doesn’t mean everyone else will. I certainly wouldn’t think the Pinot was better than the Cab if I were enjoying both with prime rib.

    It’s always a crapshoot if you haven’t tasted the wine yourself. Sure, you have a fairly good idea if you know how a critic’s likes and dislikes compare to your own, but it’s never a guarantee.

  13. Mark, good point re: Moscato. I should have said my palate reflects the informed consumer’s preference.

  14. SUAMW, incorrect. You have strayed far from the truth. Metaphor and analogy are central to human communication — to art — to conversation. Try getting through a day without using them.

  15. Tom, “does relative value play a role in assessing the quality of a wine?” No. But it does count in (a) the verbiage accompanying a score and (b) whether or not the wine gets a “Best Buy” or “Editor’s Choice” special designation.

  16. I think this is going to be good, because it gets to the heart of appreciation. To what degree is someone “objective”, merely spotting quality, and to what degree are they showing personal bias? The wine world is full of issues like this around tannins, acid balance, use of wood etc etc etc.

  17. Steve

    I work hard to eschew metaphors and analogies in favor or specific and concrete words.

    It is entirely possible. One only has to try.

    Metaphors and analogies are more vague symbols than specific words are. As those more vague symbols are much more subject to individual interpretation, specific and precise words win out when one wants to communicate clearly.

  18. I believe that winemakers in the luxury segment of the market commonly craft their wines in a manner to please the critic. It’s in this segment where scores and critic’s recommendations are most important to the wine buyer. The most obvious reason is the price of the wines and the money put at risk for a wine that you haven’t tasted. There is also the insecurity of not knowing enought about wine to know the difference in this segment.

    It’s in the luxury market where the winemaker will incorporate new barrels, small amounts of Brett, high alcohol, late harvest fruit-forward winemaking, and other characteristics to make the wine stand out in a line up. Or if not tasted blind, the winemaker wants the critic to visit, to taste the critic through the cellar, telling the critic what he or she wants to hear, natural techniques, indigenous yeasts, biodynamic farming, yada yada yada. The goal is to impress the critic with their sincerity and subtlety. The winemaker is selling more than the wine in the glass. Here the bottle has to be big and heavy.
    It’s the story that is most important in the luxury market. In the luxury market the winemaker will always deny they are influenced by anyone other than their own personal taste. It is the most important part of their story.

    But in the low priced segment the winemaking is directed at the customer. Here you even find wineries that use consumer panels to judge the acceptablity of a new product. You almost never find Brett, oak is usually subtle, wines are under 14 alc. The wine style is one that the winemaker thinks the customer will like . The story is important here, but it is a different story directed to the wine drinker, not the critic.

  19. Morton, thank you. In my job, I cover the entire spectrum of what you describe. This is why the “luxury market” wines often score equal to or lower than the “low priced segment.” I don’t care about price and since I taste blind it wouldn’t matter if I did. I love it when a $20 Napa Cab out-scores an $80 Napa Cab. So in my case, your two “different stories” coincide. Every wine has an equal chance at a high score. That is assuredly not the case with Parker, although we can only hope Galloni will now blind taste as his boss didn’t, and I trust the wine media will report on this. As for Spectator, well, if he blind tastes, it’s not evident, and I personally don’t believe he does. My constitutionally protected opinion.

  20. Morton wrote:

    “I believe that winemakers in the luxury segment of the market commonly craft their wines in a manner to please the critic.”

    I can say with some degree of certainty that the vast majority of wines in the luxury end of the market sell well. Granted they are a minority of all wine in the market.

    That said, if a luxury winemaker is selling his wine, does this mean the buyers are generally in agreement with the tastes of the critic for whom you say the winemakers are creating their wines? Or are the luxury buyers slavishly following the tastes of the critic.

    I think its the former. I don’t have the cajones to call luxury wine buyers slavish followers. It’s too demeaning and I just don’t think it’s true in any case. But if I’m right about this, then winemakers are making wines for the consumers much more so than for the critics.

  21. Tom Wark – You may be right, but I am influenced by my own experience. Here’s a true story that helps shapes my beliefs.

    A couple decades ago I was in a difficult situation where my employer was increasingly upset that, though the wines for which I was responsible were consistently scoring in the low nineties, they were falling short of the critical acclaim that my employer was seeking. Even worse, my employer had directed me to make a quantity of special wine that would sell for about $100 a bottle. (at that time $100 was a high price) It was obvious to me that a more than doubling the price of our wine without support from important wine critic would land in the market with a dull thud.

    I had known the key critics from their first days and have always practiced a routine of tasting their favorite wines to get idea of their preferences. It was clear to me that one of the most important critics had a taste high alcohol and strong French oak modified by noticeable Brettanomyces. (that “French” taste). So in crafting the blend for the introductory luxury wine, a blend well in excess of 14% alcohol was given brand new French oak and a portion of the blend was allowed to proceed through a Brett secondary, and that 15% portion subsequently sterile filtered before blended back. The wine matched perfectly the critic’s tastes, in my opinion. The wine was given a year’s bottle age so a second vintage was in the bottle at the time the first was released. Both were made to the same style. The second vintage was five times the first in quantity.

    The wine was released and a bit before the release I sent the first and second vintages to the critic. The wines were tasted, but the scores did not appear for six months both due to lag time to print and a delay in the tasting by the critic. The new luxury wine landed with a dull thud at release. Every distributor told me the same story. The wine was too expensive. The market for wines in that category was weak. I heard every excuse. I probably sold less that 50 cases in the first few months after release.

    Then the results came out the first vintage was given a 96. The phone began to ring and before the end of that first day, every case was committed…allocated to distributors, and within a bit over a month all the revenues were in the bank. Then a few months later the review of the second release came out. A second high score and a singling out by the critic . Pow! Within two weeks the second vintage was entirely gone.

    As I said, we are influenced by our own experience. I do not believe I am the only winemaker to figure out how to sell high priced wine. In fact, I think I was a little late catching on. Anyone who wants to succeed in the top end has to pay attention to the taste of the critic. High end wine is sold by the distributor and retailer on reputation and credentials. That is why you see shelf talkers with scores and people subscibe to Parker. While many winemakers will not be as mercenary as I in deliberately catering to a particular critic, the influence is there and winemakers do pay attention to the character of top scoring wines. Maybe they do unconsciously, maybe they are just in denial. I don’t know.

    Another reason I think my views are valid. I do not remember ever in my decades in the wine business of a consumer telling me they didn’t like my wines. If they don’t like my work, they usually keep that to themselves. Same with my friends. I have had thousands of complements from consumers who have extolled virtues of wines of which I was frankly a bit embarrassed. But not so with the wine critic. I have had plenty of wines that were panned.

    So if you are a winemaker and no matter what you make, you hear good things from the consumer and friends, but get either positive or negative feedback from the critic, you consciously or unconsciously move in the direction of the critic’s preferences.

    A final reason. You are sitting around a campfire and passing around a bottle of reposado. With you are a several winemakers. You are far away from the wine biz, recreating, just camaraderie, you know you can talk in confidence. You tell the story about crafting the wine for the critic. Your story then is topped one by one, by each of your campfire tequila buddies. Maybe you just have a bunch of posers like you for friends, or perhaps it is just a part of the business that is so dependent on the critic.

  22. Morton, I just don’t see why it’s such a crime to craft a wine to the taste of critics and/or the consumers you’re asking to buy your wine. A winemaker who makes a wine that HE likes (as opposed to a wine he can sell) is a very stupid, selfish winemaker who will eventually fail in business.

  23. Thomas Matthews says:


    I’m having some difficulty parsing this sentence from one of your comments in this thread: “As for Spectator, well, if he blind tastes, it’s not evident, and I personally don’t believe he does. My constitutionally protected opinion.”

    Pardon me if my inference that you are referring to Wine Spectator is incorrect. However, if you are, “Spectator” is not a “he,” it’s a group of men and women. And whether it’s “evident” or not (and how would it be?), all Wine Spectator reviews are the result of blind tastings unless otherwise indicated in the tasting note. That’s not just an “opinion”, it’s a fact.

    Thomas Matthews
    Executive editor
    Wine Spectator

  24. I think this is an incredibly interesting issue and I appreciate your calling my take thoughtful — unfortunately, I still have to insist that you’re totally wrong on this. You admit it’s true that wineries are designing their products to please the critics, so now the only question is how much weight we should accord to the opinions of such critics who tell us that those wines keep getting better and better. Don’t you think it’s obvious that if a few hundred wineries started changing the way they do things just to appeal to your personal tastes, you’d think quality was on the rise? And isn’t it equally obvious that you’d wouldn’t exactly be in a position to make that determination objectively?

  25. Keith, you can believe whatever you want. It’s a free country. I don’t agree with you, but whatever.

  26. Not to steer the lively discussion in another direction, gentleman, but as a former retailer, I find the average consumer isn’t too concerned with points and/or is confused by the point system. The store I used to work for featured many wines rated 90 points or above by a variety of well-known sources (Parker, WS, and Tanzer included), but I found that the majority of buyers relied on advice from sales staff rather than shelf talkers. This was mainly because we could put it into terms that the average customer understood (, i.e “this wine is full-bodied and has notes of chocolate and a little ‘gum sucking’ tannin”), as opposed to somewhat obtuse descriptions included in reviews (i.e. how many consumers on average are getting hints of lilac and pencil shavings?)

    Many customers have also expressed confusion about the rating system, particularly as to why a wine like a $12 Chilean Sauv Blanc got 92 points versus 90 points for a $250 Napa Cab. My coworker came up with this helpful metaphor: Think of ratings like the Westminster Kennel Club Dog Show. You’re not looking for the “best dog” (well, not til the end, at least): You’re looking for the best poodle, the best schnauzer, the best Golden Retriever in the bunch.

Leave a Reply


Recent Comments

Recent Posts