subscribe: Posts | Comments      Facebook      Email Steve

Another wine-rating system, this time based on 1,000 points



Forget about arguing over the differences between 96 and 97 points. Now we can debate the finer distinctions between a score of 875 and 876. Or 943 and 944. Or 563 and 562. Whaaat?? That’s right. There’s a new wine rating kid in town, called Wine Lister, and it uses, not the familiar 100-point system, but a thousand point system.

No, this is not The Onion. How’s it work? Well, according to their website, they gather data from multiple sources “to give a truly holistic assessment of each wine,” and the reason for a 1000-point system is because Wine Lister “can actually differentiate to this level of precision [which protects] the nuance and meticulousness of the exercise.”

Well, yes, I suppose a 1000-point system can be described as more “nuanced” than a 100-point system. But really, people who believe in score inflation now have a powerful new arrow in their quiver with which to criticize numerical ratings. From their press release, Wine Lister seems to be using only three critics at this point: Jancis Robinson, Antonio Galloni and Bettane+Desseauve (a French-based, sort of a Wine-Searcher website).

At first consideration the notion of a 1000-point system sounds dubious. It does present us defenders of the 100-point scale a certain conundrum: after all, if the 100-point system is good, then a 1000-point system has to be better, right? Maybe even ten times better. Of course, this can lead to a logical absurdity: How about a 10,000-point system? A million-point system? You see the problem.

Of more interest to me than how many points the best system ought to have are the larger questions concerning the need for a new rating system, and the entrepreneurial aspects of Wine Lister’s owners to launch one at this time. Consumers already have many, many wine rating and reviewing sources to which to turn, both online and in print. They don’t seem to be demanding yet another one. Why does Wine Lister feel their time has come?

Well, maybe it has. Any startup is a gamble, and in the entrepreneurial world of wine reviewing, which seems to be undergoing tumultuous changes, anyone can be a winner. Antonio Galloni took a huge gamble when he quit Wine Advocate to launch Vinous, which has turned out to be such a huge success. Will Wine Lister be? I don’t know, but it has good credentials. What it has to prove is that it’s more than a simple compilation of Jancis-Antonio- Bettane+Desseauve reviews. They’re also factoring in Wine-Searcher, and there’s even an auction-value component (although most consumers won’t care about that). But beyond being a “hub of information” (from the press release), I think Wine Lister’s limitation is that wine consumers seem to want a personal connection to the recommender they listen to, which an algorithm cannot provide. I could be wrong. I’ll be following them on Twitter @Wine_Lister and we’ll see what happens.

* * *

While I am affiliated with Jackson Family Wines, the postings on this site are my own and do not necessarily represent the postings, strategies or opinions of Jackson Family Wines.

  1. Bob Henry says:

    Jancis Robinson, M.W. uses a 20-point scale. (More on this in my sequel comment.)

    Antonio Galloni uses a 100-point scale.

    Michel Bettane and Thierry Desseauve use a 100-point scale.

    I can’t see how anyone could extrapolate from any one of these scales — let alone all three — and derive scores based on a newly-fashioned 1,000-point scale.

    As for a million point scale, The Hosemaster of Wine has already cornered that market. (Parker awards bonus points for a wine’s longevity. Hoser awards bonus points for “wine payola”: kickbacks and junkets.

    Not dissimilar to the sotto voce accusations about selective British wine writers of an earlier era, who purportedly found full cases of Bordeaux in the trunks of their cars after each château visit.)

  2. Bob Henry says:

    From Jancis Robinson, M.W. Website
    (September 16, 2002):

    “How to Score Wine”

    Imagine going to an art gallery and being asked to fill in a form assigning scores to each work. It does sound pretty difficult and of questionable use, does it not?

    Yet the process of scoring wine, one which many of us engage in frequently, is not that far removed from assigning points to a Picasso or a De Kooning.

    I would be much happier in my professional life if I were never required to assign a score to a wine. I know so well how subjective the whole business of wine appreciation is and, perhaps more importantly, how much the same wine can change from bottle to bottle and week to week, if not day to day. I frequently find myself re-tasting a wine at the same stage in its life. So far I have rarely marked more than 0.5 points out of 20 differently on the two occasions, but it wouldn’t surprise me at all if I did.

    And as for tasting the same wine at different stages in its life, this is even less likely to yield identical scores. Quite apart from bottle variation there are differences in tasters’ moods and vast differences in how wines mature in bottle.

    Even I have to admit, however, that scores have their uses. The most obvious is to help the reader-in-a-hurry – and there are a heck of a lot of us around. Don’t you feel as though your life is more frenetic than it was 10 years ago? I know I do. I don’t seem to have the time to linger over my newspapers and journals. Hence the advent of arts reviews which rate even plays and films with a certain number of stars – and of course the business of assigning a numerical score to wines to help potential purchasers, however much we professionals may feel our beloved liquid is too subtle to be reduced to a single number.

    I find myself using all sorts of different scoring systems depending on the circumstances. When I taste wines for British Airways, for example (the closest thing I have to any commercial link), they have been submitted in response to a specific tender document. This means that the quality can vary from the sublime to the dire (I’m constantly amazed, and insulted, that some people clearly think we won’t notice that a wine is technically faulty or over the hill). What we’re looking for in this instance is something that is extremely sound and will appeal to as many people as possible at 33,000 feet — which generally means that the wine has to be a bit more obvious than wines we might enjoy on the ground, for our tasting equipment is far from its best in the strange atmosphere of a pressurised cabin.

    We also tend to discount wild and wacky wines which might appeal very much to a small proportion of fliers but many would find a turn-off (you can see the same, understandable policy at work in the choice of in-flight audio and films).

    We go through up to 50 wines, always blind, sometimes having split the wines into two groups for two lots of tasters, to select the best candidates and then we all taste the finalists again to decide what to buy. This means that a scoring system inherited from fellow-consultant Colin Anderson, a Master of Wine whose nose was famously insured with Lloyds of London when he was in charge of wine buying for the precursor of Allied Domecq, works extremely well. We give faulty wines a gamma (or C grade), respectable wines a beta or B grade and standouts worth buying an alpha or A. In between these of course are all sorts of fine distinctions such as B+, A- – even B++?+, but we do tend to get there in the end. This apparently rather crude system allows a group of tasters to come to a consensus much faster than we would if we all assigned numerical scores and had to get out the calculator when we conferred.

    In most of my tasting and writing I don’t really need scores. What’s important when I taste a range of mixed wines is to mark those I think good enough – which often translates into sufficiently good value (for most us price is important) – to recommend. A mere tick suffices. An exclamation mark draws my attention to something notable such as an absurdly hyberbolic claim on the back label or some strange new phenomenon. I’m sure, for instance, I gave an exclamation mark to the first Italian wine I saw labelled Shiraz, or to a curiously old vintage of a particularly commercial wine for example. ‘GV’ distinguishes the seriously good value bottles while real stinkers get a cross to bear.

    I like the five-star system used by Michael Broadbent and Decanter magazine. Wines that taste wonderful now get five stars. Those that will be great may be given three stars with two in brackets for their potential. But Brits being as polite, or just plain cowardly, as we are, almost all the wines get between three and five stars in Decanter so it’s not an especially nuanced scoring system – although I have been known to use it for wines likely to be very close together in quality such as de luxe Champagnes or mature vintage Ports.

    When even I have to admit that I really need a numerical scoring system is when tasting a wide range of wines of the same sort when readers, or subscribers to, need a shorthand reference to my favourite wines. En primeur Bordeaux, for example; early offerings from the latest Burgundy vintage; almost any horizontal tasting from producers of varying competence.

    I know that Americans are used to points out of 100 from their school system so that now they, and an increasing number of wine drinkers around the world, use points out of 100 to assess wines. Like many Brits, I find this system difficult to cope with, having no cultural reference for it.

    SO, I LIMP ALONG WITH POINTS AND HALF-POINTS OUT OF 20, WHICH MEANS THAT THE GREAT MAJORITY OF WINES (THOUGH BY NO MEANS ALL) ARE SCORED SOMEWHERE BETWEEN 15 AND 18.5, WHICH ADMITTEDLY GIVES ME ONLY EIGHT POSSIBLE SCORES FOR NON-EXCEPTIONAL WINES — an improvement on the five star system but not much of one. (I try when tasting young wines to give a likely period when the wine will be drinking best, so I do cover the aspect of its potential for development.) [CAPITALIZATION used for emphasis. ~~ Bob]

    But, perhaps strangely for someone who studied mathematics at Oxford, I’m not a great fan of the conjunction of numbers and wine. Once numbers are involved, it is all too easy to reduce wine to a financial commodity rather than keep its precious status as a uniquely stimulating source of sensual pleasure and conviviality.

  3. This one goes to 11.

    Actually, the 1,000 point system makes sense for what they’re trying to do. Keep in mind that they are not measuring *quality*, but rather desirability of a wine by incorporating quality, brand metrics and pricing. So to use the existing 100 point system would be confusing. Additionally, assuming they understand that their market is trade and maybe a handful of wine investors, and not mainstream wine buyers, then creating (false) precision is also a good product/marketing decision.

    Having said that, establishing authority in wine is a very, very long-term game. Nobody’s going to flock to abstract measurements like this. Maybe they can throw some machine learning at the underlying data to identify pricing anomalies, but they need a lot more historic data and deeper product/producer/region attribute data than they to do a good job at that.

    I wish them luck.

  4. This is simple normalization of data. Various independent data streams crammed into a database, normalized by WineLister to create a common score.

    How they normalize the data is their unique attribute to the scoring world and debate. They can choose to weight the data or score evenly or unevenly, like a financial index, say S&P 500 or Russell. It would be more helpful to have MORE data streams to create a better sample set of data, as the data they are using is limited.

    As the press released mentions, the scores from certain publications really don’t venture into the 50 or 60 point range anyway, so the bottom end of the scale of 1000 is still going to be little used due to the population of data sources…unless WineLister weights data.

    Showing a standard deviation of scores for a particular category, say Napa Cabernet, along with median, average, and range within a certain year or years, would be interesting.

    One could then create efficiency ratings between whites and reds and across the globe, similar to a Sharpe Ratio in financial statistics.

    Then sort and rank scores by critic, magazine, or crowd source and I might be intrigued by the data.

  5. doug wilder says:

    I actually stumbled across a 2 week free trial of the application yesterday but didn’t do anything with it. The infographic contained on that page shows the different core parameters used by the algorithm to derive the number. Yes, it is complicated. One of these is the state of ‘liquidity’ which seems like it would vary with supply. Without knowing how much that component is weighted, a 875 may later turn into a 922, or 814. Ultimately this system is short hand for the auction buyer who doesn’t want to do their own research. Auction is where Lister’s focus has been concentrated in her previous writing. I am sure more questions will arise once the site launches.

  6. HahahahahahahahaHA! Thanks for the laugh this morning Steve!

    Imagine — a 1000 point wine!

  7. Bill Stephenson says:

    As any winemaker receiving an 89 complains that their wine is worthy of a 90, under this system a score of 899 will lead to the same gnashing of the teeth and (assumed) loss of sales

  8. David, quality is simply an *input*… it’s not what they are measuring. Again, I think a 100 point system would confuse people because they would assume they’re measuring quality. Anything is preferable to 100 points in this case.

    Doug, it makes sense that their target is auction buyers. The problem is that it’s likely that the composite metric is a *very* poor predictor of auction results. The only way to establish credibility is to publish their backtesting results and publish predictions with actual realized prices. I have no idea who they are, but I will wager that the performance of their model is going to be embarrassingly bad.

    (On a side note, as a consulting gig I’m building an analytics system for collectible sports cards auctions and it is substantially the same problem – finding price dislocations for auction buyers/sellers… except I’m working with ~ 50 attributes, 8+ years of time series data representing millions of auctions, and some moderately fancy machine learning models. Calculating common-sense composite metrics is easy. Using them as accurate predictors of performance is mostly not much better than a guess.)

  9. doug wilder says:

    Bill, I don’t think the application of the critical score will be the only component applied to the algorithm to simply scale by a factor of ten.

  10. Bob Henry says:

    Quoting from The World of Fine Wine magazine “Contributors” Section:

    “Ella Lister has long been passionate about wine, but only after several years in finance did she make her professional foray into the wine industry. She is now a full-time wine writer and consultant, the experience she gained as an investment banker at Lazard in London bringing a unique and sophisticated insight to her take on fine-wine investment. Ella read modern languages at St. Hilda’s College, Oxford, and her love of France and Italy is equal to her love of wines from those countries. She was recognized as Emerging Wine Writer of the Year in the 2013 Louis Roederer International Wine Writers’ Awards.”

    She was a humanities major in college. Not a “quant” (mathematics or physics or engineering major).

    She worked in investment banking before following her bliss into wine as a writer.

    Citing his LinkedIn profile, their Director of Technology Henry Woodsend graduated from University of Bristol with a Bachelor of Science (BSc), Mathematics and Computer Science. His has spent his professional career in Information Technology (most recently developing mobile apps).

    As the late astrophysicist/popularizer of space exploration Carl Sagan observed: “Extraordinary claims require extraordinary evidence.”

    Maybe if someone with real science chops like Leonard Mlodinow [*] were associated with this venture, I might take interest.

    But not these two folks . . .


Leave a Reply


Recent Comments

Recent Posts