subscribe: Posts | Comments      Facebook      Email Steve

Wine rating systems: time for a change

6 comments

I spent the better part of 30 years living and working in 100-point land: the wine-rating system used by my two former employers, Wine Spectator and Wine Enthusiast, as well as by Robert Parker’s Wine Advocate.

The 100-point system surely is the most popular in the world. It has survived decades of often fierce criticism. Critics said it was arbitrary and capricious, that it presented itself as scientific when it was anything but, that it had a deleterious effect on wine style because the most powerfully extracted, oakiest wines got the highest scores. All these things were true, but the 100-point system proved remarkably robust. When I retired from formal wine tasting eight years ago, it dominated the market, and, as far as I can tell, it still does.

The 100-point system looks like it’s here to stay, at least in America. There’s nothing looming on the horizon to replace it. Oh, sure, a new generation of wine drinkers has increasingly turned to peer-reviewing on social media; they no longer care what some (usually white) wine critic says, and that’s fine. But in that sense, the market may be ahead of the industry. Winery P.R. communications continue to tout high scores (anything over 90 points) in their campaigns. As long as that’s the case, wine samples will continue to be mailed to wine critics, who will continue to publish reviews using the 100-point system, which will continue to be touted by winery P.R. people, and on and on…It’s a cycle, and like most cycles, it’s hard to stop.

But a new development in China throws all this into an interesting perspective. Mike Veseth, the respected wine economist, just published an issue of “The Wine Economist” that reports on “China’s 10-Point Scale.” That gigantic country apparently is launching an official, national rating system of 10 points that will “score…each wine on the market taking into consideration…Chinese tastes, cuisine, and culture.” The new system is being rolled out in stages. It was introduced late last year, but The Drinks Business publication reports it “is not yet compulsory for all wines sold inside China [and] may serve as a base for formulating a national [wine] recommendation system.” That article quoted a Chinese expert as predicting that, eventually, “[the] majority of wines sold in China will adopt this system.”

Now that I’m not living and working in 100-point land, I have the benefit of hindsight about the 100-point system that provided such a nice job for me for so long. And the more I think about it, the sillier it seems to be. I used to be quite sincere when people asked how I could determine the difference between, say, 87 points and 88 points.. I would say, “Easy. To me, it’s obvious.” And I could go into great detail, if they wanted. At the same time, I always admitted that, if I tasted the same wine (from different bottles) on separate occasions, chances were good that I’d give it different scores. But, I argued, in general the scores would be close together. In the end, I always said, a wine review ought to be looked at as the taster’s impression of that wine, at a particular moment in time, and consumers were free to accept, reject or ignore the review.

Nowadays, I often cringe when I see how wine scores are used. There are so many critics across this land (and elsewhere) that a P.R. person has her pick of dozens of reviews to use in an advertisement. We, the consumer, often don’t know the qualifications of the reviewer, or the circumstances under which he reviewed the wine (blind? Open?), nor do we always know with precision what the relationship is between reviewer and winery. Has the reviewer been paid? These are important considerations. (Of course, the new Chinese system suffers, I would think, from the same drawbacks.) I turn to critics and scores to inform my own buying decisions, but I always feel a little guilty about it. I wish that all numerical rating systems would go away, and be replaced by something more esthetically satisfying: a short essay, for example, that showed real writerly qualities.

I think there’s a place for more intelligent, nuanced wine reviewing. As we emerge from the pandemic, it’s going to be a different world. After all these months of sheltering in place, people may well be more reflective, and less reflexive. I know that social media tends to work in the opposite direction, making people think less; but here and there I pick up on clues that younger people are getting tired of social media. They’re reading more books and spending less time scrolling through meaningless Twitter feeds. I’m hoping to see new publications emerge that treat wine consumers as intelligent, thinking adults, instead of like cows lining up for silage.


Do red wines get higher scores than whites due to “bias”?

3 comments

 

Is there “a critical bias toward red wines” among wine critics? That’s the thesis of a thought-provoking study that examined 64,000 scores from leading publications and found some fascinating tendencies:

  • reds score higher than whites
  • red wines are over-represented above 90 points
  • whites are over-represented below 90 points

So pronounced were these findings, the authors write, that, as the score crosses the critical 90-point threshold, “selling price and selling price variation increased quickly…[with some] lower-rated reds costing more than more highly-rated whites.” For example, a 90-point Napa Cabernet might cost $75 whereas a 93-point Chablis might go for $45.

I came across an article about the study at Jeff Siegel’s Wine Curmudgeon blog. (Sorry, I don’t think the full study is available online, although it is on PDF.)

Siegel found the finding curious: “Something is going on,” he wrote. I agree. But what could it be?

Siegel himself postulated various explanations. Critics may rate red wine higher “because it’s more prestigious.” This leads to a cascade of results: Producers invest more money making red wines than whites “…because consumers are willing to pay for that prestige,” and that greater investment in the production process may result in better wines.

During my decades of being a wine critic, I thought about this topic intensely, although I never reached any definitive conclusions. But it’s pretty obvious when you consider that at the leading wine periodicals, there are more (often far more) 100-point scores for reds than for whites. (This was true for me, too. I never gave a perfect score to a white wine.)

Let’s consider the question of bias, or preconceptions. If you know you’re tasting, say, First-Growth Bordeaux or Grand Cru Burgundy or Sauternes for that matter, from a great vintage, you’re more likely to yield to the possibility of giving it 100 points than if you’re tasting, say, a Temecula Tempranillo. So, to eliminate that bias, we taste single-blind. But even if you don’t know the individual bottles, if you’re a professional wine critic and your tasting was set up by a staff person, you’re still most likely going to be told the general category. “We’re tasting Premier Cru red Burgundy today from the 2011 vintage,” or “This flight consists of 2013 Napa Valley Cabernets and Bordeaux blends under $40.” Armed with these telltale bits of information, the brain will begin to come to certain conclusions, albeit unconsciously: a below-$40 Napa Cab cannot possibly get 100 points (so the reasoning goes); the best it can aspire to is 96, maybe 97 points, and so that’s what the critic finds when he tastes the wines.

So let’s make the tasting double-blind: nothing is known about the wines except for the color. This is where the bias for red wines (if there is one) comes in. You cannot prevent the critic from knowing the color. (You can always use black glasses, but I know of no critic who routinely uses them in assessing wines.)

The more I think about it, the more I believe there is a bias toward red wines, and I think Siegel stumbled upon the truth. Red wine is perceived as “more prestigious.” To understand why, you have to look at history. The French invented the system of categorizing wines by status (Grand Cru, First Growth and the like), and they tended to reserve their highest categories for red wines. In turn, the British fundamentally invented the game of writing about and critiquing wine, in the eighteenth and nineteenth centuries; and they overwhelmingly favored French red wines over whites. They therefore gave their highest plaudits to red wines. Our American and British systems of wine reviewing today—from Oz Clark to Robert Parker—are direct descendants of those British wine writers of yesteryear. The inherent bias toward red wines has filtered down over the centuries and still exists.

Which begs the question: Are red wines actually better than white wines? Well, there is the argument they’re more complex: more skin and seed contact, more oak (usually, at the high end), and so on. Does more complexity = “better”? That’s a hard case to prove. At some point, what we know, or think we know, about wine gets so inextricably bound up with the pure and simple physical experience of tasting it, that it’s impossible to separate the two. Which, come to think about it, is perhaps what makes wine so great: its pleasure is as much intellectual as hedonistic.


If point scores give you reassurance, go ahead and trust them…

3 comments

 

…no matter how many articles like this one you read that tell you to ignore them.

Now, the first thing I’m going to tell you is that the author of the article, MJ Skegg—a good writer–got all nine of his bullet points correct! MJ is the wine writer for the Portland, Oregon, Mercury, and yes, he’s right, for the most part, when he makes his accusations against scores:

  1. It’s all subjective
  2. Wine critics are human
  3. The wines start to look the same
  4. Experts are inconsistent
  5. They ignore context
  6. They inflate prices
  7. The scores keep getting bigger
  8. The system is (allegedly) corrupt
  9. They’re prescriptive

I might dispute some of his points a little, and I will in a second; but by and large he’s correct (although he’s not really breaking any new ground. Other writers and bloggers have made the same points for years). So how come I say that, for all the correctitude of his points, they still are not (as the lawyers say) dispositive?

Because you could say the same things about any system of wine reviewing! Go down the list and substitute any system you want; each of them is capable of being critiqued for all nine of MJ’s reasons. So that means no system is better or worse than any other. You might as well pick and choose the one that works for you. That any system of judgment created by humans is fallible is obvious; that doesn’t mean we should throw the baby out with the bathwater.

Besides, we all know that, of all the reviewing systems in the world, the 100-point system is the most popular. Like the old saying goes, fifty million Frenchmen can’t be wrong. Therefore, if you’re using it (and I do, when I’m looking for a wine), you shouldn’t feel guilty.

I will admit, as I have before, that MJ’s point (e), “scores ignore context,” is true. It’s hard to pack context into a number! However, every point score I’ve ever seen, including my own, also had a text review attached, which is where you’ll find the context. Granted, a 40-word text review isn’t very capacious, and I always found myself wishing I could write 100 words, or even more, for my reviews; one could write a book on some wines. But you have to draw the line someplace. In one of his articles, MJ’s reviews sound just like they came from Wine Enthusiast, only without the number! Not much context there. I also don’t quite “get” the accusation that scores inflate prices. Not sure how that works. Wine prices have been going up (like prices for everything else) since, like, forever. Take a peek at Eddie Penning-Rowsell’s “The Wines of Bordeaux” to track classified growth Bordeaux prices over centuries. Robert Parker did not create the demand for the First Growths; it’s been there since before America was a country.

So I would tell consumers, Hell, yeah, MJ’s brief concerning scores is spot-on. But rather than undermining scores, he actually makes the case for them, and for the wine critics who use them. Critics are human, just as MJ points out. They are fallible; they have their foibles; neither are they consistent. But don’t you want a human giving you their take? They, like you, me and MJ, are just out there, doing their jobs. If you find a critic you can relate to, at least you know whom you’re dealing with, as opposed to crowd-sourced-type reviewing platforms, which are a mobocracy. If Steve Tanzer or Paul Gregutt floats your boat—if you know them (or feel as if you do) through their writings—if you trust them—if you understand that, as MJ implies, point scores are figurative rather than literal, and you know how to use them as part (but not the whole part) of your buying decision—if you feel that you can use all the help you can get in making that buying decision (and don’t we all?)—then go right ahead, use point scores. Like I said, when I’m exploring a wine or region I’m not that familiar with, I always turn to my trusted bevy of 100 point-based critics, and I’ve not often been disappointed.

* * *

Sorry for not posting yesterday. I’m in Oregon. These travel days don’t leave a lot of extra time for creative writing, and I don’t want to put up crap.

 


A Sauvignon Blanc tasting that raises questions about point scores

7 comments

 

We had a perfectly lovely blind tasting yesterday, 12 Sauvignon Blancs, six of them from Jackson Family Wines wineries, and the others from around the world. It was a bit of a hodgepodge but I just wanted to assemble a range that showed the extremes of style, from an Old World, low- or no-oak, high acidity, pyrazine-driven tartness to a bigger, richer, riper New World style of [partial] barrel fermentation. Here, briefly, are the results. The entire group of tasters was very close in its conclusions—a highly-calibrated group where we achieved near consensus.

My scores:

94 Matanzas Creek 2014 Sauvignon Blanc, Sonoma County

93 Robert Mondavi 2013 To Kalon Vineyard Reserve Fumé Blanc, Napa Valley

93 Matanzas Creek 2013 Journey Sauvignon Blanc, Sonoma County

92 Stonestreet 2013 Alexander Mountain Estate Aurora Point Sauvignon Blanc, Alexander Valley

90 Merry Edwards 2014 Sauvignon Blanc, Russian River Valley

89 Peter Michael 2014 L’Apres-Midi Sauvignon Blanc, Knights Valley

88 Jackson Estate 2014 Stitch Sauvignon Blanc (Marlborough) NOTE: This is not a Jackson Family Wine.

87 Francois Cotat 2014 La Grande Cote, Sancerre

87 Arrowood 2014 Sauvignon Blanc, Alexander Valley

87 Cardinale 2014 Intrada Sauvignon Blanc (Napa Valley)

86 Goisot 2014 Exogyra Virgula Sauvignon Blanc (Saint-Bris)

85 Sattlerhof 2014 Gamlitzer Sauvignon Blanc, Austria

The JFW wines certainly did very well, taking 3 of the top 4 places. The surprise was the Matanzas Creek Sonoma County—it’s not one of the winery’s top tier Sauvignon Blancs (which are Bennett Valley, Helena Bench and Journey) but the basic regional blend. But then, I’ve worked with small lots of all Matanzas’s vineyards, and know how good the source fruit is. This is really a delightful wine, and a testament to the fact that great wine doesn’t have to be expensive. It’s also testament to the art of blending.

But I want to talk about the Francois Cotat, as it raises important and interesting intellectual considerations.

The Cotat immediately followed the Mondavi To Kalon, always one of my favorite Sauvignon Blancs, and the first thing I wrote, on sniffing it, was “Much leaner.” Of course the alcohol on the Cotat is quite a bit lower, and the acidity much higher: it was certainly an Old World wine. But here was my quandary. In terms of the reviewing system I practiced for a long time, this is not a high-scoring wine; my 87 points, I think, is right on the money. It’s a good wine, in fact a very good wine, but rather austere, delicate and sour (from a California point of view). I could and did appreciate its style, but more than 87 points? I don’t think so.

And yet, I immediately understood what a versatile wine this is. You could drink and enjoy it with almost anything; and I was sure that food would soften and mellow it, making it an ideal companion. Then I thought of a hypothetical 100 point Cabernet Sauvignon that is—let’s face it—a very un-versatile kind of wine. It blows you away with opulence, and deserves its score, by my lights. But the range of foods you can pair it with is comparatively narrow.

So here’s the paradox: The higher-scoring wine is less versatile with food, while the lower-scoring wine provides pleasure with so much. It is a puzzle, a conundrum. I don’t think I’m quite ready to drop the 100-point system as my tasting vernacular, but things are becoming a little topsy-turvy in my head.

* * *

While I am affiliated with Jackson Family Wines, the postings on this site are my own and do not necessarily represent the postings, strategies or opinions of Jackson Family Wines.


Another wine-rating system, this time based on 1,000 points

10 comments

 

Forget about arguing over the differences between 96 and 97 points. Now we can debate the finer distinctions between a score of 875 and 876. Or 943 and 944. Or 563 and 562. Whaaat?? That’s right. There’s a new wine rating kid in town, called Wine Lister, and it uses, not the familiar 100-point system, but a thousand point system.

No, this is not The Onion. How’s it work? Well, according to their website, they gather data from multiple sources “to give a truly holistic assessment of each wine,” and the reason for a 1000-point system is because Wine Lister “can actually differentiate to this level of precision [which protects] the nuance and meticulousness of the exercise.”

Well, yes, I suppose a 1000-point system can be described as more “nuanced” than a 100-point system. But really, people who believe in score inflation now have a powerful new arrow in their quiver with which to criticize numerical ratings. From their press release, Wine Lister seems to be using only three critics at this point: Jancis Robinson, Antonio Galloni and Bettane+Desseauve (a French-based, sort of a Wine-Searcher website).

At first consideration the notion of a 1000-point system sounds dubious. It does present us defenders of the 100-point scale a certain conundrum: after all, if the 100-point system is good, then a 1000-point system has to be better, right? Maybe even ten times better. Of course, this can lead to a logical absurdity: How about a 10,000-point system? A million-point system? You see the problem.

Of more interest to me than how many points the best system ought to have are the larger questions concerning the need for a new rating system, and the entrepreneurial aspects of Wine Lister’s owners to launch one at this time. Consumers already have many, many wine rating and reviewing sources to which to turn, both online and in print. They don’t seem to be demanding yet another one. Why does Wine Lister feel their time has come?

Well, maybe it has. Any startup is a gamble, and in the entrepreneurial world of wine reviewing, which seems to be undergoing tumultuous changes, anyone can be a winner. Antonio Galloni took a huge gamble when he quit Wine Advocate to launch Vinous, which has turned out to be such a huge success. Will Wine Lister be? I don’t know, but it has good credentials. What it has to prove is that it’s more than a simple compilation of Jancis-Antonio- Bettane+Desseauve reviews. They’re also factoring in Wine-Searcher, and there’s even an auction-value component (although most consumers won’t care about that). But beyond being a “hub of information” (from the press release), I think Wine Lister’s limitation is that wine consumers seem to want a personal connection to the recommender they listen to, which an algorithm cannot provide. I could be wrong. I’ll be following them on Twitter @Wine_Lister and we’ll see what happens.

* * *

While I am affiliated with Jackson Family Wines, the postings on this site are my own and do not necessarily represent the postings, strategies or opinions of Jackson Family Wines.


« Previous Entries

Recent Comments

Recent Posts

Categories

Archives