Blind tasting at Wine Spectator again an issue
Once again the charges are flying concerning if advertising influences scores at Wine Spectator. I have no reason whatsoever to think it does. If I’m in a position to claim that Wine Enthusiast doesn’t let advertising influence scores — and I am in that position — then it would be churlish of me to think the opposite when Tom Matthews asserts, as he does, that there’s a firewall between the advertising and editorial departments at his magazine. It’s true at Wine Enthusiast, just as Tom asserts it is at Wine Spectator, that “our advertisers frequently complain about their ratings,” although I don’t think in our case it’s “frequently” so much as occasionally. (I may be shielded from some of their complaining.) I do know Wine Enthusiast’s advertising team is sometimes disappointed with my scores. But they’ll be the first ones to tell you, “Hey, we can’t tell Steve what to do” and thankfully they don’t try. They know that once a magazine loses credibility it’s done for.
After I Ieft Wine Spectator, around 1993, I got a job as an editor at the first Wine Business Monthly. Then-editor Lewis Perdue gave me an assignment to interview a guy in Sonoma County who claimed he’d done a computerized statistical analysis of Wine Spectator box ads and scores and found a positive correlation between the size of the ad and the rating. Lewis asked if I would also get a response from Marvin Shanken.
Marvin took my call. I explained the background, then asked if I could ask him a few questions. He exploded and hung up. A day or so later I got a letter from Wine Spectator’s lawyer threatening legal action against me if I reported the study. So did Lewis. Needless to say, being in no position to defend ourselves against Marvin’s attorneys, we decided in favor of discretion and killed the story.
In Tom’s reply to the Wine Economist story, he wrote: “…every review of a newly-released wine is the result of a blind tasting, where neither producer nor price is known by the taster.” That’s a pretty unequivocal statement, and it made me wonder about it for 4 reasons.
1. I’ve been with Jim Laube when he tasted wines openly and took notes.
2. I’ve been reliably told that at some regional tastings Jim tastes openly.
3. Certain winemakers have told me that when Jim visits with them he tastes openly.
4. On the rare occasions when Wine Spectator let me sit in (and weigh in) on formal tastings, the panel knew the general tier of the wines, e.g. new premier cru white Burgundy. So they may not have known the specific bottle prices but they had a pretty good idea.
So I emailed Tom Matthews to ask about these anomalies and he replied, in part: “…any editor can taste any wine in any setting, blind or non blind, for informational purposes…But such ‘reviews’…are not published in our Buying Guide.” To which I replied: “If Jim did in fact taste and take notes with these producers at their wineries, does that mean he did so strictly for informational purposes, and then later reviewed the wines formally under blind conditions?” To which Tom replied: “…yes, if he was tasting wines at wineries, or in other public settings, he was taking notes for informational purposes only, and the wines were subsequently officially reviewed in our normal blind tastings.”
Tom said he’d have Jim get in touch with me, but so far he hasn’t. If and when he does, I’d ask Jim who sets up his tastings (if anyone) and what he knows about the wines (vintage, variety, region, general price bracket, etc.). It’s hard — no, make that impossible to think he (and the other Spectator critics) know nothing about the wines they review except their color. Still, the issue (of blind or not) is irrelevant to that of advertising affecting scores. I am absolutely convinced it doesn’t at Wine Spectator. That potboiler ought to be permanently put to rest.
Readers can make up their minds how they think Wine Spectator critics taste. I can tell you how I taste. Sometimes blind. Sometimes not. Sometimes semi-blind, in that (having no staff) I’ll set up my own tasting so I know in advance what wines are included, but at the time of tasting I don’t know which is which. But I’ll tell you something that may surprise you (and irritate some people). I don’t think it matters how a good taster tastes. We have fetishisized blind tasting into a religious cult, just as we’ve demonized open tasting into a sin. There are no simple answers, just simple questions.