Excerpted from Thinking, Hard and Soft

What you say isn’t always what you mean. It is one thing to talk about the probability of an event or about how much you enjoyed your dessert wine. It is another thing to commit to a number consistently. What odds are you willing to accept? How does this wine or stock rate against the others that you evaluated today?

Professional wine critics write and rate. The word cloud above represents a selection of the most frequent words in about 8,000 wine notes written by 12 popular wine writers. It gives an impression of what is said.

Most of these wine notes also include a score. No matter how many words are in the note, this is what it boils down to. 98 points from Robert Parker can make a wine’s price rocket, 72 points won’t.

A wine writer might argue that you can’t really separate a score from its note. The score is justified by the note and vice versa. In an interview, wine critic Neal Martin said:

I don’t see them on their own. When you meet people who are very anti-scores they will take one note and say, “How can it be this?” But I’m writing that score as part of a tasting note, having tasted all the wines from that grower…

Still it is interesting to ask how close the relationship is. How large is the gap between what wine writers say and what they mean?

High or Low?

Let’s see if we can build a simple model that takes in words and spits out ratings. To make this a bit easier, let’s not try and predict exact number ratings, but simply whether a rating is high or low. To define high or low, I pick the threshold score that separates the wine notes into two equally sized sets. Every score above or equal to the threshold is high, everything else is low.

So we take the wine words in a note and use them to predict whether a score will be high or low using a decision tree model. Each node represents a word in the corpus of wine notes. To predict high or low, start at the top and ask: is the word “full-bodied” in the wine note? If it is then the count of “full-bodied” is 1, which is larger than 0.5 and we move to the right hand side of the tree. If it isn’t then we are in the left-hand side.

Suppose “full-bodied” is not in the note and that “medium-bodied” isn’t either. This means we are down to the “decades” node. If “decades” features then we move down to the right and hit a “True” leaf, which means we predict that the note is associated with a high rating.

Based on this model, we summarize this writer’s word to score relationship:

Full-bodied with no hint of mediocrity is high, but a mediocre attribute must be balanced with something great. A medium-bodied wine is always low. Otherwise, if it will last for decades, is massive, stunning, brilliant… thumbs up! 

This is a naive model, but it does reasonably well, correct for just over 75% of test cases.

Sooner or Later?

Rather than predicting the score, let’s predict the optimal estimated drinking age. We split the notes again, this time into wines that have been stamped “drink soon”  and wines earmarked for long-term storage.

Unsurprisingly “decades” comes out top this time.

If “decades” doesn’t feature, there is still hope. A full-bodied wine with no hint of mediocrity is set for the cellar. A full body that is backward needs time to develop. If the years are mentioned more than once, a dark wine should be drunk sooner rather than later, otherwise a massive wine should keep.

The model performs very similarly to the regression tree for wine scores, correct for 75% of test cases.

Next stop?

The approach is naive, but it gives you a feeling for the gap between what a writer says and what a writer means. No two writers will be alike and you can check this by comparing their trees. Some writers are less predictable than others. I’d see this sort of analysis as a tool to help professional critics understand their own criticism. The question is: once you’ve seen your tree, will you lose sight of the forest? Once you tailor your words to match your rating, you might as well stop writing!