subscribe: Posts | Comments      Facebook      Email Steve

How I score wine

29 comments

It comes in. I taste it in a flight of similar wines, insofar as possible. Let’s say they’re Napa Valley Cabernet Sauvignons. Since I have no staff, I know, perforce, what the wines are in advance, because I’m the one opening them and setting them up. But I don’t know what they are at the time I’m tasting. I might know that among them is Caymus Special Selection, but I don’t know which bottle.

(Anticipating the inevitable objections, let me say that no critic I ever heard of knows absolutely nothing about the wines he’s tasting. That would be a double-blind tasting. My tastings are single-blind. I like to have a little context. We can talk more about this later.)

I open the bottles about 1 hour before the session begins, to give them a little air. Let’s say I have 12 wines. I’ll pour the first six into identical glasses, then let them breathe for a few minutes. (I’ll repeat the process for the second six.) Then I sniff each, one after the other, forming my first impressions. No notes yet, just mental quick takes. “This one smells like classic Cabernet,” or “pretty oaky,” or “smells vegetal.” Things like this form the preliminaries to a formal comparative ranking.

By then, I have tentative favorites. Then it’s on to the mouth! I’ll sip each of the six, being careful to spit. Once in the mouth, I can confirm, or not, the initial impressions formed during smelling. That one with classic Cab aromas? Maybe it’s classic in the mouth, but then again, maybe it’s not, and turns overly tannic and astringent. Here’s where the mental calculations get more complex. Let’s say that Cab is classic all the way, from aroma through finish. I can’t find any flaws. I’ll push it to the front of the group — it becomes the pace-setter for the rest. Maybe I’ll give it a tentative “96.”

From here on, it’s a matter of re-smelling, re-tasting, re-calibrating, thinking, comparing and refining. Once I determine I really want to give the classic Cab 96, everything else falls into place. The Cab that’s almost as classic, but not quite, gets 92 or 93. The Cab that’s vegetal gets 84. And so on.

Why 96 and not 97? For one thing, I give almost no 100s, 99s or 98s. I have, and can, but I want to keep such exceptional scores rare, so that when they do occur, they’re taken seriously. Some magazines have “score inflation,” which I don’t like. A score of 96 is really the highest “regular” or everyday score a wine can aspire to, in my system.

Okay, so the classic Cab gets 96. Why does the vegetal Cab get 84? Well, on Wine Enthusiast’s rating guidelines, a score of 84 means a wine that’s “good,” i.e. sound and proper, without notable technical flaws, or at least without large flaws. Vegetal smells in a Cab lower a score, but I don’t consider them flaws. I could have given the wine 83 (the lowest “good” rating), but that seems overly harsh. I could have given it 86, but 86 verges on “very good” (87 points), and a vegetal Cab is certainly not “very good.” An argument can be made for 85 points; I admit, this is a judgment call. My judgment is 84, so 84 it is. Could it have been 85 on another day? Certainly.

At some point, I hand-carry the glass with the wine to my computer, where I fill in the actual score and text into Wine Enthusiast’s database. This is the final act of the tasting process, where everything is confirmed. When I hit the “send” button, the review is transported to the magazine’s server, in New York, and my part in the ratings chain is over.

I hope this gives readers more of an insight into how I determine numbers. Yet I suspect those who don’t like numbers won’t be convinced by anything I write, anyway.

numbers

  1. Thanks for the insight, Steve. Quick question though, do you feel that method leaves some wines to a matter of chance based on the setter/pacer score? If a wine is lucky, it might end up in a group where it becomes from the front runner compared to the others and then everything else is gauged lower accordingly. But then, how would that front runner compare to the front runner in another flight? If that front runner were in another group filled with other front runners would it all of sudden rank a 92 instead of 96, while another remains a 96? I know it’s impossible and for the most part, it must be an overwhelming task. I don’t mean to nitpick faults, I’m more so curious on your thoughts regarding that.

  2. Thanks for this insight into your tasting process. It’s always very interesting for me to hear how exactly this is done, and I appreciate your transparency.

    If you don’t have an tasting assistant, it seems like it would be hard to conceal the identity of the 12 wines you’re tasting. Could you explain a little more how you keep track of which wine you’re tasting without revealing to yourself the identity each wine while you’re tasting it?

  3. You are doing a fine Atlas-like job of hoisting the entire weight of justifying the 100-point scale onto your own shoulders. Who can argue with your detailed explication above of what you taste and how you sort if out? It’s all quite sensible, but it does not negate a few very simple facts:

    1) This is YOUR approach to the subjective task of rendering your impressions in both words and numbers. The use of numbers remains a function of personal, subjective standards and personal methodology. The fact that other 100-point critics use different methods and apply different definitions of quality/correctness still means that the system itself has no possible standardization, even within one magazine, let alone across the landscape of buying guides.

    2) Nothing you describe here will prevent the lazy or space-constrained marketer or retailer from snipping the rating off your review. Removed from your description and devoid of any context regarding what an “84” or “87” means in the W.E. scale, the wine becomes just a number that someone else will interpret based on what he/she thinks and 84 or 87 means. Again, through no fault of your own, the system fails despite your efforts.

    3) In insisting on using numbers, or at least working within a system that insists on them, you’re short-selling the value of your own skills and experience and effort. Can’t a review stand on its own words? Proof that it can and does can be found every week at your other reviewing outlet, The Yummy Letter. In fact, I just got fresh edition in my email box today, where I can read about a $10 Riesling and $90 Pinot Noir. http://www.yummyletter.com/newsletters/sip-i22.html#recomendation You do a great job of characterizing what these wines are like, when they might best be used, and the fact that you recommend them. And it’s all done without numbers. Magic? Nope, just a professional written critique.

  4. Steve,
    Thank you for explaining how you rate a wine. While I personally don’t lay a lot of weight in the numbers, but rather the word crafting of your tasting notes, it is nice to know how you arrive at a specific quantitative rating. I also like that you say that if you give an 84 vs. an 85 it is your judgment. Ultimately, isn’t that what it is all about? The judgment of the consumer? As a retailer I know that most “regular Joes” off the street want something quick and simple to aid in their purchasing, but being a winegeek I want more than a quick fix, which is why I look forward to your reviews.

  5. Morton Leslie says:

    Dismissing criticism as simply as, “There are people who don’t like numbers” is a pretty typical response. You change the issue from the fact that you are not acting professionally in your approach to sensory analysis and that you mislead readers into thinking you have a precision in your tasting that doesn’t exist to “there are just knuckleheads out there that don’t like numbers.”

    There are, in fact, individuals who love wine, have high standards when it comes to sensory analysis and would like to see the wine criticism elevated to a state approaching professionalism. In fact, though they will never let on in your company, the majority of winemakers I know routinely joke about this farce of 100 point scoring and the people who use it. But the joking is not laughter, but rather derision, for the damage it and those who use, it have done to winemaking in the last two decades.

    There are a growing number of wineries who have discontinued sending wines to the most prominent of these 100 point critics. The winegrowers who have done this, are among the most thoughtful students of winemaking and wine appreciation. If I were a critic, I would take notice.

  6. Robert, the wines are in bags. I shuffle them up. Then I let an hour or so go by. Then I pour and let some time go by. Then I start my sniffing. I would say that 90% of the time I don’t know which wine is which. Sometimes I do and it’s unavoidable and all I can do is try my level best to be objective. By the way, knowing the identity of a wine beforehand can be a mixed blessing. If you know it’s Latour, your expectations are heightened, so if it doesn’t meet your expectations, you feel let down and might be more critical of the wine.

  7. Oy, the weekly numbers thing again. I’m not a numbers fan, but believe that it’s perfectly acceptable and even useful to folks who need wine guidance

    Oddly, I taste wine exactly the same way Steve does, only in my underwear. I mean the wine, not me.

    Gratuitous cheap jokes are appropriate for yet another numbers “debate.” Steve, are you trying to up your average for Comments?

    I am shocked, shocked!, to see Tish arguing against numbers. And I hear Rush Limbaugh came out and said he didn’t like Obama.

  8. Dylan, this is about context, which I want to write about pretty soon. Context is all important and it’s unavoidable. It’s also about benchmarks. There are certain wines I’ve given perfect scores to (Shafer Hillside, for example) and the memory of that Cab lives on in my mind. People call it “tasting memory.” It becomes the measure against which all other Cabernets must perform. So the 96 point Cab is close to that Shafer, but not quite. Your comment is also about consistency. Can a critic be totally consistent all the time? No. I’ve reviewed the same wine twice, spaced over several months, and come to different conclusions. This is why I say that any wine review is a snapshot, a moment in time, of the critic’s impressions at that time. I like to think that I, and other reputable critics, would score the same wines more or less equally over time. But it doesn’t always happen. Usually, yes. Always, no.

  9. Ron, next week’s blog will feature an animation of Tish in his bikini briefs issuing fatwah #1,000,000,000 against point scores.

  10. I can’t resist a blast against wine critics, not just Steve, but those who insist on making a living criticizing others’ work. Writing about wine, vineyards, wineries, personalities, events serves a wonderful purpose and is a noble cause. Assigning scores to a product’s quality that is 100% subjective is just silly.

  11. Why do folks use numbers as part of wine reviews? It is because they are part of the communication. Tish sort of gets it but doesn’t really, by suggesting in his final paragraph that words work by themselves.

    The fact is that they work by themselves up to a point. And the further fact is that words are simply a means of communication. I just spent a week traveling with a great taster who hates the 100-point system. He instead uses words like “Stupendous”. What the hell does “Stupendous” mean? Is it better or not than great? How about fabulous, lovely, world-class, exceptional? When I give a wine an 87, it means I like it a little more than I like a wine I give 86. How in the name of all that is holy can that concept be construed as a joke or a failure? It is simplicity, it is purity of thought. It is an expression of my perceptions. It adds to my words, it enhances my words, it communicates to my readers.

    It is not the use of numbers that is the problem. It is their misuse whether it appears in blogs or at the retail level. Numbers or any other form of symbolic notation of quality have no descriptive meaning (zero, zip, niente) without words for context. And words can be and are enhanced by the addition of symbolic notation whether it is numbers, stars, chopsticks, puffs or Ron Washam’s briefs.

    Does anyone here doubt for a minute the objectivity, usefulness, clarity of Consumer’s Reports? Have you noticed that they also use a 100-point system as PART of their communication process with their millions of readers?

    This is all nonsense. Words have meanings, imprecise meanings in describing levels of pleasure. Even pain is measured in numbers because words have meanings but also limits. Symbols and numbers have good uses. Anyone, including my good friend Tish, who suggests that there ought to be a limit on the way we communicate ultimately wants there to be less communication, not more.

  12. Of course the topic here is way more convoluted than can be dealt with in one post and comment thread. Not unlike the twisted underwear?

    Once again (and as Steve might say, for the 1,000,000,000th time), it’s not the ratings themselves that are a problem, it is their abuse. In the context of Charlie’s CGCW, to which I now subscribe, the numbers work exactly as Charlie says they do — as part of the communication. But as we all know, the process that Steve describes in such detail above ends when he turns in his scores; but that is the beginning of the dark side of ratings.

    In my earlier comment referencing The Yummy Letter, I was trying to compliment you, Steve, on your being able to write clearly and usefully about wine without numbers. Strange how you don’t seem to want to take that as positive.

  13. Well, Tish, if you think I’m such a great writer, then just read the text part of my reviews and cover up the ratings. Oh, and buy my books!

  14. Charlie Olken mentioned Consumer Reports – I am curious to know if he or other views them as a good source of wine information? I have friends who are advocates of their wine reviews, especially the one that says wines generally tasted the same for several days after opened.

  15. Ron–

    The first time I saw wine reveiws in Consumers Reports, I said “Oh,no” because I have such high regard for the publication and I certainly did not need yet another giant publication to compete with. But, then I got to thinking and there are some missing pieces there.

    The first is that we have no idea who does the tasting or where they do it. Let’s assume that it is not done in their laboratories by their regular staffers. After that, we are left with a guess as to how and who and where and when. About the only thing I do truly believe is that the process is fair.

    Fair does not equate, however, to comprehensive, insightful, accurate, etc. Those descriptors are harder to come by for any of us, and CR is less convincing to me personally than most of the pros in the business.

    That said, their audience is not mine or Heimoff’s or Parker’s. And the wines they recommend are clearly for a general audience. I wrote newspaper columns for ten years. When I started, it was because the Food Editor at the LA Times asked me to contribute a tasting note column every two weeks, running opposite Rod Smith’s very good story line columns in the other weeks. And his instructions to me were to keep it cheap because this was a daily newspaper, not Connoisseurs’ Guide.

    He wanted no wines over $25 except for the special occasion wine. Consumer’s Reports operates further down the food chain than my columns did at the Times, and as such, it probably does not need to have the same breadth of coverage as I and other writers offer.

    So, I have come to accept that it serves a very specific audience, and while I think it should serve it in a more open and complete fashion, I suspect that its wine coverage is just what they feel they should be doing for their audience. In that regard, it is like what Dottie and John do in the WSJ. Theirs is not my kind of winewriting either, but it does a fine job of serving the audience–and that is the bottom line for any of us.

  16. I am with Tish on this one. Steve I have read your reviews and I do disregard the numerics. It is that the scores have become too important. I have, as most of you, seen this beast grow over the past decade. Steve you mentioned that 84 is a good wine, but not to an average consumer. I have heard shoppers say “I only buy 90 points or better” and they are talking about wines at $7.99. It is just silly to try to put a number on something so subjective.

  17. Well said, Steve. Thanks for the clarity you offered in your follow-up. Tasting memory is a very interesting and valid example of providing context without the, well, context. While I’m not a professional taster in any form of food or wine, I, of course, have compared certain meals with the “perfect scores” I hold deep in my mind from prior experiences.

  18. Good heavens. I thought you were going to explain how you score wine. You know, how you get it. My bad.

  19. Nancy, I don’t get your question. Can you rephrase? Glad to respond.

  20. “Score”, as in: “acquire”….

  21. I score wine… by writing a wine blog, people send me lots of it! (ba-dum-dum!).

    Steve – thanks for laying this process out here, I’m sure there are more than a few readers who don’t know how this sort of tasting takes place so it’s good to see the transparency.

    I share some of the thoughts as others who’ve commented here that the “pace setter” wine might re-align the continuum of the scores. What I mean is, it seems to introduce the possibility that if the pace setter is very good, it could move the rest of the wines up or down the scale but only in comparison to that pace setter. Which I suppose leaves it up to your experience as a taster to level-set the scores against your internal memory banks of previous tastings.

    Tricky territory, but possibly navigable if you have enough wine tasting under your belt. Can’t say I’d be comfortable doing it – but then, that’s one of the reasons why I don’t offer any type of rating system.

    Cheers!

  22. Joe, all I can say is that I have a lot of winetasting experience under my belt. In some ways I wish I were 30 again, but that’s an existential lament on the part of people who are getting older. Anyway, I see nothing wrong with the “pace setter” model. How else could you do it?

  23. Charlie wrote: “…what Dottie and John do in the WSJ. Theirs is not my kind of winewriting either, but it does a fine job of serving the audience–and that is the bottom line for any of us.” Glad you brought up this husband and wife team. Their Sat. column plays a huge role in wine criticism for the educated consumer, if not always the educated palate. They purchase enough wines from a variety of online and offline merchants in the category of the week to make general observations about the state of current releases for that category. They usually taste blind, but sometimes not, and over the course of several nights they’ll pick their five or six favorites which fall at the higher end of what amounts to a 10 pt system, using well understood terminology: good/very good/delicious. They make the point that their picks are their picks and represent what they like in an Australian Shiraz or American rosé at its best. They also augment their prose with informal videos, an important part of the new media.

  24. Steve, like umpteen people above me have stated, thank you for explaining how you approach tasting wines. Interestingly, i realized early on if i can’t smell it , i probably can’t taste it either. The fact you smell all wines, then let them sit, come back , re-sniff, then taste is of interest. I will put the sniffing first and then coming back and tasting into my routine. Also interesting how you get your first impressions, and it sounds like these are subconsciously guiding you. Whatever, thanks for the simple, straight forward “How to” by S.H. Your earlier blog on 100 pt scale helped more as to ratings. The technique and routine help more w/r/to helping me go about consistent tasting technique. I know this is simple stuff to you , but thanks for writing it up.

  25. Steve, you forgot the obligatory step of the process during which the database craps out, and you send me an email in frustration ;)

  26. Do we ask movie reviewers if their impressions are statistically significant? Do we want to debate the “thumbs up” scoring system? What about a restaurant reviewer? Do we want to know what they ate last to see if there is a bias? Maybe they had spicey food for lunch and now their palate is not in proper shape for a seafood restaurant. Holy mackeral.

    I don’t think Steve or anyone is presenting the reviews as sensory analysis or there would be labs acoss america rating wines, not magazines.
    Just having a number there doesn’t make it a quantitative analysis by any means. They are always just opinions of people with educated palates and I think most people know that. It is a guide that can be helpful particularly once you know a reviewers preferences.

    I know which movie reviewers I usually agree with and which ones I don’t. That doesn’t mean that all reviews are worthless.

    I think this was helpful and shows the sincere effort taken to sample the wines.

  27. Erika, so true. Alas…

  28. Steve, you write: “At some point, I hand-carry the glass with the wine to my computer, where I fill in the actual score and text into Wine Enthusiast’s database. This is the final act of the tasting process, where everything is confirmed. When I hit the “send” button, the review is transported to the magazine’s server, in New York, and my part in the ratings chain is over.”

    Can I ask a stupid-simple question: When do you actually do the tasting note?

    I ask this at the risk, again, of being branded only anti-ratings. But seriously, in this whole harangue about how you find a pace-setter and come up with the numbers, you refer to the characteristics you find in the wine, but I re-read this post and see nothing about when/how you do the actual description that I, a word guy, care about.

    Based on this, is it correct to assume that the rating actually — and always — comes first in your methodology?

  29. Yes the rating comes first. I do the tasting note “in my head,” then compose it for the computer. A tasting note is like a haiku. It requires some effort to get clarity and precision into 30 or 40 words.

Trackbacks/Pingbacks

  1. wineconversation.com » Blog Archive » Interactive wine tasting - [...] How I score wine (steveheimoff.com) [...]

Leave a Reply

*

Recent Comments

Recent Posts

Categories

Archives