The Wall Street Journal ran a great article recently about the flaws and inconsistencies in wine ratings, based on a study conducted by Robert Hodgson, a retired professor who taught statistics at Humboldt State University.

In his first study, each year, for four years, Mr. Hodgson served actual panels of California State Fair Wine Competition judges—some 70 judges each year—about 100 wines over a two-day period. He employed the same blind tasting process as the actual competition. In Mr. Hodgson's study, however, every wine was presented to each judge three different times, each time drawn from the same bottle. The results astonished Mr. Hodgson. The judges' wine ratings typically varied by ±4 points on a standard ratings scale running from 80 to 100. A wine rated 91 on one tasting would often be rated an 87 or 95 on the next. Some of the judges did much worse, and only about one in 10 regularly rated the same wine within a range of ±2 points.

This is not surprising at all to me. I've been saying for years that wine ratings are subjective. Different people, different tastes.

A Hint of Hype, A Taste of Illusion [WSJ]