CoffeeWars 4: Results
OK, this is what you really came for, isn't it?
Judges were asked to rate the brews in a variety of categories
on a scale from 1 to 10. For the purposes of scoring, all non-
integer results were rounded up to the next integer. Also,
people who submitted out-of-bounds scores (like 0, or 10+) were
corrected to the nearest legitimate rating. All derived statistics
(such as average or standard deviation) are based on the number
of valid entries: if a judge left a category blank, this did not
count either for or against the entry; it merely increased the
weight of each score that was received.
Bang-for-the-buck (highest overall score for an entry, divided
by the price per pound) was only calculated for those entries
that included cost information, obviously.
Two entries were disqualified:
- Baby's Jet Black (entered by Dean):
The considered opinion of the judges was that this
coffee was flavored. Flavored coffee is not allowed.
The entrant protested, and produced real live
marketing literature from the roaster indicating
that this was an unflavored brew. This was,
therefore, a controversial decision, and we had
misgivings about it: nonetheless the decision
stood.
- Jacobs Kronung (entered by Wolfgang aka seele):
The intent of the rules was to permit only entries
in bean form, and this was pre-ground. It would
have been disqualified on that basis, except that
another entry, which was clearly within the spirit
of the rules (because it was home-grown!) arrived in
that form also. So we were going to try it, until
someone spotted the Kraft Foods logo on the bag (a
nearby German speaker confirmed that we were not
misinterpreting the label). This pushed things over
the edge: CoffeeWars is about small-batch artisan
roasts, not mass-produced corporate offerings. Sorry,
Wolfgang. Maybe Kraft makes excellent coffee in Europe,
but we wound up disqualifying your entry.
Truthfully, we would rather not have had this happen. It's not
our objective to bar anyone from entering the competition, but
we had way more coffee than we could accept, even without bending
the rules to allow more in. Next year's announcement will be more
explicit about what constitutes a valid entry, and we apologize to
those who went to the trouble of bringing their coffee, only to
have it go unsampled.
Entry #13 (Prince of Darkness) deserves some special mention. The
entrant, JP (not that JP), was visiting
his favorite roaster, Oso Negro, in Nelson, British Columbia. That's
in Canada, for those of you following along at home. He described
CoffeeWars to them, and they got so excited that they donated his
entry. This, truly, was amazing: our first entry backed by corporate
sponsorship and our first indication that
CoffeeWars is known abroad. Amazing, and exciting. Thanks, Oso Negro,
for being so cool! Also: it was good coffee!
Overall Results
This table presents the various entries in order of average overall
score. In the event where overall scores tied, "Taste" was used as
a tiebreaker category. There is one mysterious entry, to which the
ballot form seems to have disappeared. It didn't place too highly,
so I don't think there is cause for concern, but nevertheless, I
wish I knew what it was. The ballot submitted in Jay Dyson's name
may have been for two entries; if that's the case, this is most
likely the first of the two.
| Coffee Name | Entered By | Score |
| Jamaican Peaberry |
Hook |
8.3 |
| Ground Zero Papua New Guinea |
Sweetypig |
7.1 |
| Oso Negro Prince of Darkness |
JP |
6.7 |
| Sulawesi Peaberry |
Rob Nielson |
6.7 |
| Abyssinia & JBM Wallenford Estate |
Jay Dyson |
6.7 |
| Kona |
Rob Nielson |
6.4 |
| Lavazza Qualita Oro |
Gurney & SpyBoy |
6.4 |
| Caffie Lobbo |
Abby Normal & Annoney Moose |
6.3 |
| Indian River Roastmaster's Choice |
Cheshire Catalyst |
6.0 |
| Guatemalan Queen |
DaMan |
5.9 |
| Deep Snow |
Cindy |
5.7 |
| Ancora d;Oro |
Foofus |
5.6 |
| Kona |
Tweakt |
5.3 |
| Silver Canyon French Roast |
Tim |
5.3 |
| Baby's Private Buzz |
Dean |
5.1 |
| Mystery Entry |
(is this the other half of Jay's?) |
4.9 |
| Hot Lava Java |
Dr. Vann Harl |
4.8 |
| Havana Gold |
Dr. Vann Harl |
3.7 |
For better or worse, the individual comments made by the judges
have been omitted. Sometimes they were inscrutable (such as
"meh." or "shiny!"). Sometimes they were insightful ("nutty bean
flavor."). Sometimes they were harsh ("is this water?"). Likewise,
they sometimes contradicted the scores on the ballots (one judge
made a comment praising the excellence of the entry's aroma,
and then rated both bean and brew aroma low; another wrote
that the coffee was very good, but then gave it low marks overall).
I don't know what to make of this information, except that we will
endeavor to make the scoring process a little more rigorous next
year. One metric of accuracy, perhaps, is the standard deviation,
and we should strive to supply guidelines where it grows so large
that it encompasses, say, 50% of the entries.
Bang-For-The-Buck Results
This table presents the various entries in order of our bang-for-
the-buck metric. Bang-for-the-buck is a derived statistic, calculated
by dividing the overall score by the cost per pound (in US dollars).
Entries for which no price data was submitted are excluded, of course.
In the event of a tie, the raw overall score was used as a tiebreaker.
| Coffee Name | Entered By | Score |
| Ground Zero Papua New Guinea |
Sweetypig |
1.0 |
| Ancora d'Oro |
Foofus |
0.8 |
| Guatemalan Queen |
DaMan |
0.7 |
| Deep Snow |
Cindy |
0.7 |
| Oso Negro Prince of Darkness |
JP |
0.6 |
| Silver Canyon French Roast |
Tim |
0.6 |
| Indian River Roastmaster's Choice |
Cheshire Catalyst |
0.5 |
| Kona |
Tweakt |
0.4 |
| Baby's Private Buzz |
Dean |
0.4 |
| Jamaican Peaberry |
Hook |
0.3 |
Truly, this is interesting data. First, the overall winner was
also the lowest score, in terms of bang-for-the-buck. This just
goes to show you that you get what you pay for, but there may be
a point of diminishing returns. Similarly, the second place entry
overall was also the highest bang-for-the-buck. This just goes to
show you that excellent coffee does not have to cost an arm and
a leg. Sometimes just an arm is sufficient. Or a leg. Whichever
you'd miss less.
Miscellaneous Other Results
In other categories, the winners are as follows.
| Category Name | Winning Entry |
Entered By | Score |
| Bean Aroma |
Jamaican Peaberry
| Hook |
8.7 |
| Brew Aroma |
Jamaican Peaberry |
Hook |
8.4 |
| Brew Strength |
Jamaican Peaberry |
Hook |
7.3 |
| Brew Taste |
Jamaican Peaberry |
Hook |
8.3 |
| Brew Acidity |
Jamaican Peaberry |
Hook |
7.8 |
There's a moral here, too: the winning entry was the winner overall,
in all categories. There wasn't just one score that tipped the scales.
Some have hypothesized that the reason that entry received
such off-the-scale marks was just that it was the last entry to be
sampled, and the judges were simply going crazy with the emotional
release of finally finishing their hard work. Perhaps, to some
small degree, this is true, but I think not greatly. This was some
outstanding coffee, and it richly deserved to win.
OK. That's it for the results summary. All the data is available
for download from this site, in case you want to do your own
calculations.
This page is maintained by Foofus.
Please send any comments to foofus at foofus.net.
