Finca Nuguo Beat Hacienda La Esmeralda By Just One Fifth of One Point. How Common Are Such Results?
[crossposted from the OCN blog]
I’ve been a judge on a few international cupping panels, in Africa and in Latin America, specifically. One thing you notice when you spend a lot of time around competitions like ACE’s Cup of Excellence and AFCA’s Taste of Harvest, is that the difference in quality between the very best coffees is razor thin.
For instance, here is a snapshot from the Specialty Coffee Association of Panama, showing the winning coffees from the 2015 Best of Panama competition. These are the top rated lots from the Exotic Washed category, which is probably the single most competitive category of any country in the world right now.
You can see the final average scores printed out here according to mean, median, and mode. Mean is used to determine the winner. The coffees were still coded at the time, so no one knew which was which (except for any hunches that the judges might have had in their minds). After the winners are determined blindly, the head judge has the lab technicians reveal which code corresponds to which coffee.
You can see that he jotted down the “reveal” here so that he could tell his fellow judges what was going on. Within a few hours after this stage, the official list would have been compiled in a neat and tidy way for the official announcements. So this photo shows you a little window into how things look behind the scenes at a competition like this.
You can see that the difference between the top coffee, the “Ju-Mix” from Jose Gallardo at Finca Nuguo, beat out the second place coffee, from Hacienda La Esmeralda, by just under 0.2 points. With a panel of eight to twelve judges, which is typical for such competitions, 0.2 points is basically statistically insignificant.
But there has to be a way to determine a winner, of course. So even with a razor thin margin, a winner is declared.
I’d also point out that the mode for Jose Gallardo is 93.5, which means that it was the most commonly awarded score by the judges. In a group of this size, I would guess that it means 2 or 3 judges gave it exactly that score.
The mean is also higher than the median. That means that there may have been one or two judges scoring significantly lower than the main cluster of judges.
Incidentally, I was not on this panel, but I had the chance to cup all the coffees that went into Finca Nuguo’s winning blend about a month before the competition, and I scored them from 89.5 at the lowest (to be fair, it was a coffee that was still very young and volatile); to 96.5 at the highest, which is the highest score I’ve ever given out in my life.
I pulled some numbers from comparable competitions, the Cup of Excellence competitions that have been held in Latin American with washed coffees since the beginning of 2015. You can see the margins between first and second place here.
A couple of things to notice here.
- First of all, the biggest margin is still less than one point! Most of the margins hover right around 0.5.
- The margin between Finca Nuguo and Hacienda La Esmeralda is the slimmest margin on the list.
- The scores in Panama are higher overall. This is interesting, but should be taken with a grain of salt, because it was run by a different organization than the other competitions on this list. Nevertheless, it still fits with the general impression that Panama is the elite of the elite when it comes to world coffee quality
Clearly it’s extremely competitive at the top. And these hair’s-breadth margins continue on down the line. For example, the difference between second place and third place in the El Salvador CoE was just 0.15.
It’s another interesting question to ask, “What does a difference of 0.2 taste like?” I would like to address that question in another post, as it’s probably a little bit controversial. In any case, it’s a very fine distinction.