“Context-sensitive” wine tasting
Mike Veseth, the wine economist whose blog posts and occasional research papers I always look forward to reading, published another thought-provoking piece last week.
It’s about what he calls “context-sensitive” tasting, a term I’d never heard before, but one I’ll use going forward, because it’s snappy and useful.
My readers know that I tend to be obsessed with the mechanics of tasting: single-blind, double-blind, open. As a Gemini (not that I believe in astrology, but the characterization of Geminis as being able to see things from multiple points of view certainly applies to me), I can appreciate the pros and cons of all three approaches.
Double-blind really makes you think hard and deep. Single-blind gives you just enough context to judge whether or not the wines are good examples of their class. And open tasting gives you the complete context that can make the wine-drinking experience so multi-dimensional.
The problem with these three approaches, or at least the first two (single- and double-blind) is that they’re fundamentally incompatible with consistency of judgment. The same wine, tasted blind on multiple occasions, will impress even the most professional reviewer differently, leading to questions concerning reality. What is the wine really like? It’s not really like anything; it all depends how you experience it.
(Of course, if you’re tasting openly, you can be 100% consistent. You can also be 100% biased based on what you know you’re tasting!)
This is why Veseth titles his post “In Vino Veritas?” with the question mark playing a pivotal role. The conventional wisdom, among the general public who read wine critics, is that there’s something “real” or “true” in wine that critics are in a unique position to perceive and describe. Well, there is something “real” in wine, but it’s not what critics perceive and write about, it’s what a wine laboratory measures with instruments. You can determine everything from aluminum to zinc in wine. Those things are “real” and “true,” but they obviously are not things critics look for, nor are they things most consumers care about.
Thus, Veseth writes, “our impressions of wine is [sic] context-sensitive–perhaps more so than we really want to admit.” This conclusion chagrins him, but it shouldn’t.
So is there “reality” in wine, beyond a lab analysis of its chemical and physical properties? No. In that case, proponents of open tasting have a point when they say, in effect, “Why bother tasting blind if you know the results are not replicable?” And furthermore: “Since open tasting is reliably the most consistent method of tasting, it also is the most trustworthy.
Well, yes…but then, why was it so hard to get anyone from the Wine Advocate to state, in no uncertain terms, that they do taste openly, instead of dancing around the issue for so many years? To the best of my knowledge, it wasn’t until Antonio Galloni told me that he does (you can read his quote here, from last year)
that anyone from the Advocate organization. And Antonio did so not only with candor, but with passion: he strongly defended tasting openly at high-end properties, whether in Burgundy or the likes of Harlan Estate. Why, then, does TWA’s website, presumably written by Robert Parker, say, “When possible all of my tastings are done in peer-group, single-blind conditions (meaning that the same types of wines are tasted against each other and the producers’ names are not known.”? Admittedly, this is Parker talking…he pointedly says “my” tastings, not “our” tastings, so maybe he was implying, very gently, that not every review in TWA is the result of a “peer-group, single-blind” tasting.” Or maybe standards have changed since he wrote those words.
Well, I don’t mean to criticize Antonio’s method,or anyone else’s, since, as I said, each approach has its strengths. In the end, consumers have to decide what methodology they want their critics to use. To tell you the truth, I don’t think most consumers care. But they should. If all the wine critics in the world tasted blind, the hierarchy of pricing and tiers would fall down and crack, like Humpty-Dumpty, and all the King’s men couldn’t put it back together again.
So what’s my solution? If I’m limited to just one form of tasting, I prefer single-blind. The way around this is transparency. Every review that anyone does should have a little symbol next to it: TO (yasted openly), SB (single-blind), DB (double-blind). That way, consumers who cared would know, and if someone was concerned with bias, they could dismiss a TO wine.
as a long-time sceptic of the 100-point scale, i was rather impressed that the WA, WS, and WE all gave our 2010 pinot noir the exact same score.
And, Gabe, what does that tell you?
Does it tell you (cynically) that all reviewers are alike?
Does it tell you that the 100-point system has magically become man’s best expression of wine quality?
Does it tell you that you no longer need to read the words of those reviews? I would be much more impressed if the organoleptic analysis were the same rather than the scores.
I will tell you, because you asked, what it tells me. It tells me that expertise in wine evaluation is important. It tells me nothing about the value of the 100-point system because that system is nothing more than a shorthand expression of the more detailed findings expressed in the written part of the reviews.
And, frankly, Gabe, and I apologize for sounding critical and cranky, but your comment tells me that the trees and the forest are being confused. Your comment suggests that the 100-point system is reality, when it is no such thing. It is only a snippet of reality–a shorthand for a qualitative conclusion and absolutely useless absent the words that folks like Steve and Parker and Tanzer and I pen with great care. The truth is in the words, not in the numbers.
So, Gabe, apologies again as I have read your commentaries here and elsewhere and have appreciated your insights. I just do not get this one. BTW, I use the 100-point system because it does not hurt my opinions to append a notational shorthand to them.
It has nothing to do with the methodology, but the person who is doing the reviews that we trust or don’t trust. Good post!
gabe: We did that at our secret meeting.
Great post on a delicate subject. “Context” can also mean what other tasters had said about a wine. This is one of the probs. with the 100-pt scale: Once I learn what somebody’s score was for almost any wine, I find it hard to dismiss it from my mind as I drink, and it thus forms part of the context for my reaction.
Charlie,
i wasn’t really trying to make any strong points, other than to say that wines scores might not be as random as people like you and i think. beyond that, all i can say is that my opinions of the 100-point scale have drastically evolved since it became my own hard work that was being critiqued, and those opinions are still evolving.
Steve,
how do i get an invite to one of those meetings? can you at least save me a bottle of Willams Selyem or Kosta Browne?
Gabe
I think you made a very strong point, and now you have made it again. While there may be no dispute in matters of taste, the pros who do this for a living actually do know what they are doing most of the time.
I happen not to like the 100 point system but any system of symbolic notation has its virtues and its shortcomings. And among those virtues are the ability to sharpen the intent of the words we use to describe wines. I think your increasing acceptance of the system is as much borne of your acceptance of the usefulness of critics as anything else.
Yeah, I do think an acceptance or appreciation of critics is part of it. Another big part of it really is my appreciation for the score itself, and not just the review. I think of it like getting a grade on a test. Getting an A doesn’t make you smarter than a person who gets a B, but you can’t just ignore your report card.
It’s not that I don’t think tasting notes have value (especially for consumers who haven’t tasted a wine before), but I don’t need someone else to tell me what my wine tastes like. I’ve tasted it many more times than they have, and I know everything about it down to the last detai. Really, it’s just nice to have a third party verify the quality of your wine. Sure the Wine Club loves it, but what do the pros think?
The best example I can give you is our 2012 pinot noir, which has been called the best Willamette Valley vintage in decades, but has been the most challenging vintage in my very short winemaking career. I will spend many more weeks working and sweating and worrying about this wine. When it’s done, it might be great and it might be terrible. I can tell you already, my wife will love it and I will think its flawed. It will probably sell at the same rate that all of our pinot sells. I really want to send that wine in to be rated, and not to help sales or read tasting notes. I want to know if its good. While one score may not give me a definitive answer to that question, a few scores will certainly give me a clearer picture of the wines overall quality.
Alright, sorry to go off on a rant, but you prodded me a bit 🙂 Thanks Charlie
When it comes to reviewing wines, I have respect for any method that delivers consistent, transparent and unbiased results. I learned to select wine by tasting openly. Every wine critic is going to have different preferences or experience but as my friend Charlie (the most rigorous taster I know), pointed out the number is not the only information to pay attention to in a review (If I didn’t need to write a couple hundred tasting notes my job would be a lot easier!) At the end of the day, we are usually not that far apart in a final assessment. Check out the Blackbird Vineyard site to see what I mean. http://www.blackbirdvineyards.com/news/reviews.php
Aw shucks, Doug. You make me blush. And thanks.
I think that almost all of us who do this for a living are pretty rigorous. That is why I find the label-open tastings that Galloni, Parker and Tanzer do at wineries to be so contrary to the standards that I believe we should all follow.
Charlie, since it’s widely known that the critics you cite taste openly, then why is there not any backlash by consumers and the trade?
We have seen minor moments of protest, but my guess is that the trade prefers open tasting as it gives them an advantage.
And my further guess is that most consumers do not know and really are not versed in the niceties of open vs single-blind vs double blind.
We know the retailers don’t care because their focus is on selling wine and they are not arbiters of technique. They are arbiters of advantage in selling, which means that they follow the names that move wine as their single measure of meaning.
Charlie, in your comment: “We know the retailers don’t care because their focus is on selling wine and they are not arbiters of technique. They are arbiters of advantage in selling, which means that they follow the names that move wine as their single measure of meaning.” doesn’t take into account those retailers who taste, and buy ahead of the curve because they trust their own palates and know what their consumer wants. That is the way i approached it as a retail wine buyer. I don’t dispute there are still those retailers who run to the phone to call the broker as soon as a critic gives a wine a high rating. They usually come up empty.
Fair point, Doug.
Those retailers would not care about the rigor or non-rigor of reviewers because they are their own reviewers.