“Proof by ethos” is a term from Artistotle, referring to a method of persuasion, by appealing to a speaker’s authority and credibility. In science, according to a recent paper [more on this later], it refers to a situation in which “a scientist’s status in the community is so high that everybody else takes this person’s calculations or results for granted. In other words, nobody questions the validity of that scientist’s claim because of the particular ethos that is associated with that person.”
It is thus more or less identical to the more familiar Latin term, argumentum ad verecundiam, or “argument from authority,” which is often cited as a major potential fallacy in argument: One cites an “authority” to prove one’s position, but in that particular case the so-called “authority” is wrong, so the person citing him also is wrong.
Both concepts—“proof by ethos” and “argument from authority”—are natural to humans. None of us can know everything; we need to trust others to inform us about things we don’t know, or don’t understand sufficiently. It is, in fact, one of the glories of humankind that we are social and trusting enough to turn to the advice of others, sometimes for things that impact our very lives, on the very fragile basis of belief. So“proof by ethos” and “argument from authority” are not bad in themselves. But they must be taken in context: if the “high status authority” we listen to is mistaken, or deliberately misleading, all kinds of bad consequences may ensue.
The recent scientific paper I referred to, concerning the age of the Earth, points out how easy it is even for trained academicians to succumb to the perils of “proof by ethos.” The famous Nobel laureate, Richard Feynman, once postulated, in a lecture that was transcribed, that the center of the Earth must be younger than the Earth’s surface by “one or two days” due to the relativistic effects of gravitational time dilation.
(Read the paper yourself. You can skip over the mathematical formulae and pass from the Introduction to the Discussion and Conclusion. Fascinating, paradoxical stuff.)
However, the paper’s authors discovered, through simple back-of-the-envelope calculations, that “years” should have been substituted for “days.” The center of the Earth, that is, is one or two years younger than its surface, not one or two days. “[E]ither the lecturer [Feynman] or the transcribers had it wrong,” the authors concluded. Feynman died in 1988.
The paper’s authors decided that any of Feyman’s colleagues or even his grad students could easily have discerned Feynman’s mistake (if, indeed, it was his and not the transcribers’). That they did not, and for so many years, is a prime example of the danger of “proof by ethos.” Nobody realized that the “days” citation was wrong, because everybody implicitly trusted Feynman so much that it seemed silly to second-guess his conclusion.
We see, in wine reviewing, much this same “proof by ethos” or “argument from authority.” Substitute the words “wine critic” for “scientist” and the opening quote in this post becomes “a wine critic‘s status in the community is so high that everybody else takes this person’s [reviews] for granted.”
That’s how it works, isn’t it. We’ve reached a situation in which we—the collective “we” of the wine industry and marketplace—largely accept the “truths” of the critics because, after all, “they”—the critics—possess an “ethos” that places them beyond doubt. So haloed are the critics in the glow of their own infallibility (or the infallibility we impute to them) that their pronouncements have the power of the edicts of a shaman. It is the particular quality of humans that we elevate a select few from our own midst to this priesthood. Unsure of ourselves, irresolute in how to negotiate the world, we confer high status upon them, and then lay our belief at their feet. Only humans need gods. I’ve been that wizard behind the curtain, though, and I can assure you that even the critics, the high and mighty, have feet of clay, and will remain “gods” only so long as “we” so elect them.
If you’re one person, No. A single taster will always be tasting within the parameters of his limitations, e.g. he may be more or less sensitive to TCA than other tasters. He may wince at the smell of pyrazines, or find the heat from alcohol unbearable, or feel that a totally dry wine is too severe.
But how about a group? Can the dynamics of consensus solve the subjectivity dilemma?
Objective tasting has been the unicorn of the wine industry for centuries. A long time ago, it was assumed that an epicure, like Thomas Jefferson, was correct in anything he said about wine. Nowadays, in our era of mistrust of authorities, we no longer take it for granted that anyone can be the supreme expert. “Galloni might not like it, but I do,” the reasoning goes—as it should.
But sometimes, it’s important to understand exactly what you’re dealing with in a wine. Is it really balanced? Is it really dry? Is it reduced? What do we mean by “creamy” or “rich” or “spicy”? These are the kinds of things two tasters can easily disagree about, sometimes violently; but if you have a group, you can more easily arrive at a consensus. Or so the theory goes.
My own approach to these matters has been based on my experience as a wine critic. I’ve said for years that, if you’re a consumer interested in wine, then find an expert you trust, and stick with him. (And it doesn’t have to be a critic. It can be a merchant, or your sister-in-law.) In other words, find someone whose palate you relate to, and trust.
But there is something to be said for a group consensus. We’re all part of a group: the human race, and moreover, of a sub-group within it: American wine consumers. Group influence, AKA peer pressure, can be strong, especially when people are as unsure of wine as most people are. And—just to underline my point—everyone is unsure of his or her palate: not just ordinary consumers, but critics, winemakers, even, dare I say it, Master Sommeliers. Everyone seeks refuge within the safe harbor of a peer group. It’s the herd instinct that makes, for example, impalas cluster together when lions stalk the perimeter.
Whether you go with group consensus or individual reviews, is up to you. It depends on your purpose. But I do think that, if you go with the group, you should make sure your group knows what the heck they’re talking about. These crowd-sourced reviews, where anyone can weigh in no matter what their professional qualifications, are questionable to me. Does that sound anti-democratic? Pro-elitist? I guess it does. But I do think reviewers need to bring credentials to the table.
* * *
While I am affiliated with Jackson Family Wines, the postings on this site are my own and do not necessarily represent the postings, strategies or opinions of Jackson Family Wines.
Every day, I get blast email advertisements from wineries or wine stores touting the latest 90-plus point score from Suckling, Parker, Vinous or some other esteemed critic. Here’s an example that came in on Saturday: I’m reproducing everything except the actual winery/wine.
_____ Winery’s ____ Napa Red Wine 2013 Rated 92JS.
Notice how the “92JS” is printed in the same font type and size as the name of the winery and wine. That assigns them equal importance; the rating and critic are virtually part of the brand. Later in the ad, they have the full “James Suckling Review” followed by a full “Wine Spectator Review” [of 90 points]. This is followed by the winery’s own “Wine Tasting Notes,” which by and large echo Spectator’s and Suckling’s descriptions.
Built along similar lines was a recent email ad for a certain Brunello: The headline was “2011 ____ Brunello di Montalcino DOCG”; immediately beneath is (in slightly smaller point size), “94 Points Vinous / Antonio Galloni.”
We can see that, in these headline and sub-heads, through physical proximity on the page or screen, the ads’ creators have linked the name of the winery and the wine to the name of the famous critic and his point score. One of the central tenets of advertising is to get the most important part of the message across immediately and strongly. (This is why so many T.V. commercials begin with the advertiser’s name—you hear and see it before you can change the channel or click the “mute” button.) In like fashion, most of us will quickly read a headline (even if we don’t want to) before skipping the rest of the ad. The headline thus stays in the brain: “Winery” “Wine Critic” “90-plus point score.” That’s really all the winery or wine store wants you to retain. They don’t expect you to read the entire ad, or to immediately buy the wine based on the headline. They do expect that the “Winery” “Wine Critic” “90-plus point score” information will stay embedded in your brain cells, which will make you more likely to buy the wine the next time you’re looking for something, or at least have a favorable view of it.
This reliance of wineries and wine stores on famous critics’ reviews and scores is as strong as ever. There has been a well-publicized revolt against it by sommeliers and bloggers, but their resistance has all the power of a wet noodle. You might as well thrash against the storm; it does no good. The dominance of the famous wine critic is so ensconced in this country (and throughout large parts of Asia) that it shows no signs of being undermined anytime soon. You can regret it; you can rant against it; you can list all the reasons why it’s unhealthy, but you can’t change the facts.
Wineries are complicit in this phenomenon; they are co-dependents in this 12-Step addiction to critics. Wineries, of course, live and die by the same sword: A bad review is not helpful, but wineries will never publish a bad review. They assume (rightly) that bad reviews will quickly be swept away by the never-ending tsunami of information swamping consumers.
Which brings us back to 90-point scores. They’re everywhere. You can call it score inflation, you can argue that winemaking quality is higher, or that vintages are better, but for whatever reason, 90-plus points is more common than ever. Ninety is the new 87. Wineries love a score of 90, but I’ve heard that sometimes they’re disappointed they didn’t get 93, 94 or higher. Even 95 points has been lessened by its ubiquity.
Hosemaster lampooned this, likening 100-point scores to Oprah Winfrey giving out cars to the studio audience on her T.V. show. (“You get a car! And you get a car! And you get a car! And YOU get a car! Everybody gets a car!”) Why does this sort of thing happen? Enquiring minds want to know. In legalese, one must ask, “Cui bono?”—Who benefits? In Oprah’s case, she’s not paying for the cars herself; they’re provided by the manufacturers, who presumably take a tax writeoff. It’s a win-win-win situation for Oprah, the automakers and the audience.
Cui bono when it comes to high scores? The wineries, of course, and the wine stores that sell their wines (and put together the email blast advertisements). And what of the critics?
Step into the tall weeds with me, reader. A wine critic who gives a wine a high score gets something no money can buy: exposure. His name goes out on all those email blast advertisements (and other forms of marketing). That name is seen by tens of thousands of people, thereby making the famous wine critic more famous than ever. Just as the wine is linked to the critic in the headline, the critic’s name is linked to the 90-plus wine; both are meta-branded. (It’s the same thing as when politicians running for public office vie for the endorsement of famous Hollywood stars, rock stars and sports figures: the halo effect of fame and glamor by association.) There therefore is motive on the part of critics to amplify their point scores.
But motive alone does not prove a case nor make anyone guilty. We cannot impute venality to this current rash of high scores; we can merely take note of it. Notice also that the high scores are coming from older critics. Palates do, in fact, change over the years. Perhaps there’s something about a mature palate that is easier to please than a beginner’s palate. Perhaps older critics aren’t as angry, fussy or nit-picky about wine as younger ones; or as ambitious. They’re more apt to look for sheer pleasure and less apt to look for the slightest perceived imperfection. With age comes mellowness; mellowness is more likely to smile upon the world than to criticize it.
Anyhow, it is passing strange to see how intertwined the worlds of wineries, wine stores and wine critics have become. Like triple stars caught in each others’ orbits, they gyre and gimble in the wabe, in a weird but strangely fascinating pas de trois that, for the moment at least, shows no signs of abating.
Years ago, during the heyday of Sex and the City, the San Francisco Chronicle ran a spoof piece on what “the girls” would be doing if they lived in the “cool gray city of love.” Samantha, you’ll recall, had her own high-end P.R. firm in Manhattan, where she represented restaurants, celebrities, clubs and so on.
In San Francisco, the Chronicle’s writer determined, Samantha would still be in P.R.—only it would be winery public relations. When I read that, I remember thinking that wine had finally and definitely come to dominate the zeitgeist. It was the cool-hot thing to do, the field everybody wanted to work in, whether in PR, writing or production.
(Sidebar: When I started out, nobody, but nobody, wanted to be a wine writer. I sometimes wonder, if I was beginning my career today instead of in 1989, if I’d even be able to get a writing job at a magazine, much less Wine Spectator. The field has become that competitive.)
Wine remains a highly coveted field for young people to work in, maybe hotter than ever, according to this article in the drinks business, which claims that winemaking and beer brewing are “among top dream jobs” for young people just starting their careers or thinking of changing. (The study was done in Britain, but there’s no reason not to think attitudes here in America are any different.)
So desirable are these winemaking and beer-making jobs that “over a third (35%) of people said they would consider quitting their job to re-train in their chosen profession – regardless of money.” That’s good, because these types of jobs typically don’t make a ton of money. Funnily enough, “Security guards (95%), IT consultants (91%) and accountants (87%) were by far the most eager to pack in the typical 9-to-5 and take up a craft career” such as winemaking.
I know people in both the wine industry and craft brewing, and most of them seem to be very happy. It’s true that the pressures can be difficult, but the joy seems to outweigh any of the inconveniences (such as basically having your normal life put on hold during crush). When I look back over my years in the wine biz, despite all the bitching and stress I went through (or put myself through), I consider myself incredibly lucky to have been able to do what I have. Coming up through the Golden Age of wine in America—the boutique era, the rise of the wine print media, the enormous popularity of wine (and beer), and the emergence of social media—has been a privilege, and also a great opportunity to see history being made, close-up, and perhaps to have been a tiny part of it. No wonder people want to work in this industry.
Have a wonderful weekend.
Did you know that I prefer organic wines to non-organic wines? I didn’t, either. But then I read this new paper from the American Association of Wine Economists, entitled “Does Organic Wine taste better? An Analysis of Experts’ Ratings,” and I found out that, yup, I do.
Well, kinda sorta. See, the paper’s authors decided to study “data from the three influential wine expert publications: Wine Advocate, Wine Enthusiast, and Wine Spectator,” and as it turned out, “During our period of study [74,148 wines produced in California between 1998 and 2009], the main tasters for California wines for Wine Advocate, Wine Enthusiast and Wine Spectator were Robert Parker, Steve Heimoff, and James Laube, respectively.”
The big P-H-L! They took our scores, crunched them in that esoteric way only economists can, and lo and behold, “Our results indicate that the adoption of wine eco-certification has a statistically significant and positive effect on wine ratings.”
How much? Not a lot: “Being eco-certified,” the authors found, “increases the score of the wine by 0.46 point on average.”
Well, one hardly knows where to begin. Right off the bat, I have a problem when the lesson that people will take away is that P-H-L (and by extension major critics) prefer organic wines to non-organic ones. Less than half a point difference? I suppose if they fed 74,148 scores into a computer and found a 0.46 point difference, then who am I to argue with HAL? But a 0.46 point difference doesn’t seem like very much to me. It’s not even round-uppable to the higher score (87.46 rounds down to 87).
But wait, there’s more. The following factors also had an impact on the scores of organically-certified wines, according to the paper:
- ” a 1% increase in the number of cases will decrease score by 0.003
- ” An increase in the number of years of certification experience by one [winery] decreases score by 0.09 point.”
Confused? I am. So the more cases wine the winery produces, the lower the score is; but the longer the winery has been certified organic, the lower the score also is!
How about the winemaker’s hair color? Did they include that?
The authors also counted the number of words in each review and found this: “Next, we examine the impact that eco-certification has on the number of words used in wine notes. As shown in regression (1) of Table 6, wine notes of eco-certified wines are not significantly longer than those of conventional wines. However, as shown in regressions (2) and (3), eco-certification increases the average number of positive words by 0.4 but has no statistically significant impact on the number of negative words.”
My interpretation of this is that it’s gibberish. The authors compiled a list of words [Table 7] but I don’t understand how they infer whether their use is positive or negative. Is “jammy” positive or negative? Do Parker, Laube and I even use it in the same way? How about “offbeat”? Is that good or bad? And “peat”: if I tasted that in an Islay Scotch it would be good, but in a Chardonnay?
The authors also state something that I don’t think is objectively true, or, even if it is, is irrelevant. “Second, as a related point, wine experts have a better knowledge about wine eco-certification and are able to differentiate between different types of eco-labels, namely organic wine and wine made with organically grown grapes, which represent different wine production processes with different impacts on quality.”
I’m not going to sit here and tell you I know the difference between different types of eco-labels. There are so damn many (different certifying agencies, “natural,” biodynamic, etc.), I get confused—and, while I’ll let Parker and Laube speak for themselves, I bet they get confused, too. Besides, if “All the publications claim blind review,” as the paper’s authors write, then we critics don’t even see the labels when we’re tasting and reviewing (much less would we have a tech sheet in front of us).
But finally, this statistic seems to be to be the last nail in the coffin of the study: “On average, 1.1% of the wines in the sample are eco-certified.” By my calculations, that’s a little over 800 wines—out of 74,148. I fail to see how you can extrapolate any useful information from such a small sample, compared to the huge number of wines in the study. Apples and oranges.
I’m no economist, it goes without saying. If I were, I guess I’d spend my days crunching numbers and coming up with interesting factoids. But I have to say, I don’t see the point of this particular study—not if it’s going to be used to make a claim that I don’t regard as true. For the record, let me say that I do not think organic wine is better. And you know what? I don’t care what the numbers say.
Ian Burrows is a great sommelier whom I first met at a Jackson Family Wines event I was speaking at. He was then working at one of San Francisco’s hottest restaurants, Atelier Crenn, in the Marina District. I was never fortunate enough to dine there, because the Marina is really a schlep from Oakland. I liked Ian a lot when we met, and he turned out to be a good correspondent, on both Facebook and my blog. So when he wrote me a fairly long comment, I took it seriously, and want to respond in kind.
Ian had read my post from a few days ago, in which I described how, in choosing wines for my tastings, I rely on—among other factors—the reviews of certain top critics. Ian wrote:
I read your article on choosing sparkling for a comparative tasting, and I have to ask, why on earth would you ever base your choices on other critics scores?
I have never understood the fascination of taking such an incredibly narrow focus on deciding which wines (or automobiles or eye-liner for that matter) are the best value, most accessible, most delicious or whatever from a handful of very influential reviewers.
Why not just send out a bunch of random e-mails to your wine buddies? Ask “what wines in XYZ category should I represent in this tasting?”…. Surely, if you spread it across continents and demographics you’d get a more accurate picture.
I have the utmost respect for what you did at WE (although I still do not completely understand it) and I have even greater respect for what you do at JFE but you gotta let go of what is, quite frankly, a waste of time….. “Wine reviews”.
Reviews – I am pretty sure they will be gone in five years.
You have a better deal being the PR front man at JFE than a reviewer because at least you can focus squarely on industry trends/changes, comment and review issues that directly and indirectly affect the quality and style of wine, not simply assign points and hope that readers respond by supporting your tastes and/or reviews.
It’s perhaps a face to face conversation for another time, but one that I know will be vibrant and respectful
I replied personally to Ian, but I want to expand on that here (and I wouldn’t be doing this if I didn’t have the utmost respect for him). My main points were, (1) I am emphatically not “the PR front man” at Jackson Family Wines! I don’t know how that rumor got started. In fact, my job has nothing to do with PR (although I suppose you could say that everything ultimately touches on public relations).
More to the point, I defend my use of other critics’ scores this way: When you’re assembling a lineup of wines for a comparative tasting, you have to use some kind of parameter. Since you can’t taste everything that theoretically falls within the scope of your tasting, you necessarily must limit the number of entries. Let me ask, Readers, how you would do it?
Let’s say, for instance, that I want to do a tasting of the Cabernet Sauvignons of Rutherford. There are at least 39 wineries in Rutherford, according to the web page of the Rutherford Dust Society. Many of them, maybe the majority, produce more than one SKU of Cabernet Sauvignon or a Bordeaux blend. Let’s say there are 100 different SKUs. That’s too many to include in a tasting, so you have to whittle down the number.
You could do this in any number of ways: Wines from west of Highway 29 on the Rutherford Bench, wines from the Mayacamas Mountains, wines from east of Highway 29 but west of the Silverado Trail, wines from east of the Silverado Trail, wines from way up in the Vacas, wines from south Rutherford, from north Rutherford, 100% Cabs, blends, wines above $75, wines below $30, and so on and so forth. Any of those would make sense, I suppose. But so does the kind of crowd-sourcing I do when I choose wines based on my own experiences, compounded by their critical scores. When Wine Advocate, Wine Spectator, Vinous, Wine Enthusiast, Wines & Vines, Wine & Food, and so on are all giving a wine high scores, that’s a pretty good indication it’s a very good wine. And those are the kinds of wines I want to include in my tastings, especially when we’re including Jackson Family Wines in the lineup. I want to see how JFW wines stand up to the most critically acclaimed wines. (And I hope I won’t be accused of wearing a PR hat when I tell you, they do very well.)
Surely Ian isn’t entirely serious when he suggests sending random emails to my “wine buddies” soliciting their views. I have about 4,000 Facebook friends and 6,500 Twitter followers. Not all of them claim to be wine experts, and frankly, I don’t know most of them, so their opinion is not of the greatest help to me. If I was doing something on popular drinking habits or trends or wine and food pairing, I might, and frequently do, ask my friends and followers, but not for assembling a blind tasting of ultrapremium wines.
Now, Ian (and a generation of young somms) may not care about the major critics—I understand that–but I do. Maybe it’s a generational thing. I respect what James Laube, Robert Parker and the others do. I know how hard the work is…what the pressures are…I know also that when you’ve tasted wine seriously for a good many years you really do develop a master palate. I don’t think there’s anything crooked or unseemly about what they do (and what I used to do). These are men and women of the highest integrity and their opinions should matter.
Nor do I think wine reviewing is “a waste of time” that will be gone in five years. I’ve frequently said on my blog that wine reviewing will always be with us, because as long as there are a zillion wines on the market, consumers are going to seek guidance. I’ve said that this guidance can come from many different sources, including a local and trusted merchant, but merchants—let’s face it—may have a motive to recommend a wine they carry, which makes them less than completely objective. A wine critic of the caliber of a Parker, Laube, Galloni, etc. has no ulterior motive. He or she doesn’t care about the advertising his publication may or may not solicit from wineries—that’s the famous “firewall” between editorial and advertising, and it’s real. Nor does the critic care whether or not someone buys something. So, unless you’re prepared to charge the critics with something untoward—and prove it—you really have no leg to stand on when it comes to criticizing them or questioning their sincerity or ability.
I will concede that every critic has his subjective preferences. Wine Spectator, in my opinion, gives too much attention to Marcassin. The San Francisco Chronicle seems to have a thing for Morgan Twain-Peterson and Bedrock. When I was at Wine Enthusiast I certainly gave a lot of love to Bob Cabral and Williams Selyem. But there’s nothing nefarious about any of this: critics are only human, and we do form attachments, to winemakers, wines and particular styles of wine.
So, my friend Ian, this is my respectful reply. I’d love to get together, anytime you’re free, to chat about this; and maybe I can explain what I did at Wine Enthusiast.
Have a great weekend!