Come on, Annie’s is Best Italian? 2010 Reader Restaurant Awards FAQ

Voting is over and the winners have been announced in the Beijinger’s 7th Annual Reader Restaurant Awards … but the talk goes on. A lot of people have lots of questions for us each year for us, and here’s a quick look at the 8 most Frequently Asked Questions about the restaurant awards.

Hey, this is all just a transparent ploy to kiss your advertiser’s ass. We all know the awards are fixed to make your advertisers win, so why bother?
We hear this one the most. However, if you take a look at the results, less than 25% of the winning restaurants are advertising clients of ours. In fact, we end up making some of our advertisers angry – those that think we have a fiscal duty to award them “Best” credentials because they buy ads in our magazine. Sure, in an ideal world we’d love it if all the restaurants in Beijing advertise (and likewise that all of our advertisers would always win their categories), but that’s not very realistic. In the end, in terms of generating advertising, the restaurant awards are a net loss for us.

We do the awards to reflect our readers’ preferences; provide a lasting guide to the city’s best that the never-ending flow of new expat readers can use as a barometer; to reward Beijing’s food professionals for doing an excellent job satisfying their customers; to promote a competitive environment in Beijing’s restaurant scene that results in rising standards across the board; and finally, to provide hot topics of discussion for our readers, who eat out nearly every day.

All the restaurants stuff the ballot boxes so it’s not truly an accurate survey.
Yup, restaurants do stuff the ballot box. But by using several key data analysis techniques we are able to filter out 99% of the ballot-stuffing attempts. Each year between a quarter to a third of ballots received are discarded for being either obvious attempts at ballot stuffing or for containing false identification details. After filtering the results, as a final step we ask for every voter to respond to us by e-mail to verify that a restaurant isn’t using their identity to cast fake ballots. Each year we add to the complexity of analysis to make sure we keep ahead of attempts at fraud.

It’s always the same winners in certain categories. Why not “retire” some restaurants who keep winning year after year?
This is a tricky one. On one hand, we look at the restaurant awards as a method of recommending restaurant choices to our readers, and if the recommendations are always the same, the awards are not really adding anything new to the discussion. On the flip side, is it really fair to a perennial winner such as Hatsune or The Tree to make the category is “Best Japanese (Other than Hatsune)” or “Best Pizza (Other than the Tree)”? It seems that if a restaurant is doing great and satisfying customers, why should they be excluded from being called the best? It’s also important to understand the span of time. Winners may not change radically from year to year, but if you look at the results from three or five years ago, there is indeed a shift in tastes and preferences.

Nevertheless, this does remain an issue, so we try to provide variety in a few other ways – first, by having Expert’s Picks, which oftentimes will vary from the readers’ picks by selecting less popular or newer venues; and secondly, by having some new categories each year (this year, new categories included “Best Contemporary Chinese,” “Best Chef,” “Best Kung Pao Chicken,” “Best Mediterranean” and “Best Tapas”).

Why have a nominating panel? It seems like it arbitrarily restricts choice.
When we first started having restaurant awards, we gave everyone blank ballots. The results were less than ideal – we found ourselves declaring restaurants “Best” by virtue of winning the vote 20 ballots to 19 (with 700 other restaurants getting two votes each). Or we’d get a 16-way tie for first place. Or we’d spend days trying to interpret hundreds of ballots that listed favorites such as “that one place near the corner of Xindong Lu, the one with the red sign.” After trying this for a year or two we realized that it made a lot of sense to have a nomination round – made up of 200 voters, half hand-picked, half randomly selected – which would allow for a distillation of top potential vote getters. Kind of like how the Oscars works. This all but assures the likely “bests” will end up on the ballot.

Why was my favorite category left out? Some really biggies were missing this year, like “Best Vegetarian” and “Best Shanghai.”
Each year we offer select 60 or more potential dining categories and offer them to our nominating panel for evaluation. By analyzing their voting patterns we can determine whether a category is worth putting into the final round. Our theory is as follows: in order for there to be a meaningful popular vote, a category must have at least eight legitimate contenders – i.e. eight reasonably good choices in the category. There can be incidents where there are too few legitimate contenders – for instance, we dropped the “Vegetarian” category from the popular vote because the nomination panel named less than eight contenders, and one of them (Pure Lotus) garnered more than 80% of the nominations. For this category, we decided it made no sense to put the category to a popular vote as the result was likely obvious. On the flip side, some categories had the problem of far too many nominees, and none were standouts – most likely a result of our readers not having a lot of familiarity with that particular category.

Isn’t this really just a popularity contest?
Welcome to democracy. To use a quote attributed to Winston Churchill, “Democracy is the worst form of government except all the others that have been tried.” In essence, voting is always susceptible to becoming popularity contest. But how else could we determine Beijing’s best? Generally popular restaurants are popular for a reason – personal tastes differ, but we think you would have a hard time saying that any of the winners of our reader’s picks are bad restaurants. They have a following because they are doing something right.

And we understand that there is often a distinction between “Most Popular” and “Best” – in fact some people will swear the two are contradictory (for example, Pizza Hut and McDonald’s are arguably the most popular pizza and hamburger joints in the city, but relatively few people would consider them “the best”). This is why we complement our readers picks each year with an expert’s panel (this year, in fact, with two separate expert panels – one consisting solely of Chinese experts, the other of international foodies) – to help provide additional choices that are less influenced by sheer popularity alone.

Don’t restaurant chains have an unfair advantage? Why not eliminate them from contention or penalize them in some way?
There was a time when there were only really two types of dining outlets in Beijing – fast-food chains and single-location eateries. But in a city the size of Beijing and with a culinary scene that is increasingly sophisticated, the multi-location restaurant has become almost the norm. A good chunk of the restaurants that are on many voters’ “Best of” lists have more than one location. To name but a few: Da Dong, Hatsune, Kro’s Nest, Din Tai Fung, Comptoirs de France, Muse, Purple Haze, 1001 Nights, Ganges, Taj Pavilion … all of which grew from one location in Beijing on the back of their successes with one location. To remove all “chains” would eliminate from contention many quality homegrown Beijing institutions. We’ve considered a cut-off such as “maximum of X locations” but each number we’ve come up with seems somewhat arbitrary. We’ve considered some sort of bonus or penalty system to try to factor in multiple locations, but haven’t perfected a formula yet.

Come on, Annie’s is the best Italian restaurant?
According to our readers, yes – and in fact has been for several years running. Is it really that big of a surprise? Annie’s is a locally born and bred Beijing institution with exceptionally reasonable prices, decent food and service and one of the city’s most efficient home delivery systems. And yes, it has multiple locations. What we’re really grappling with here is differing opinions on what is “best” – some people think only fine dining options should be considered the best; some judge on authenticity; for others it is price and convenience. Suffice it to say that our readers have provided us enough evidence over the years that Annie’s is indeed one of their favorites in the Italian category. Is it the most authentic? Is it the fanciest? Does it have the finest pasta? Perhaps not – but there’s no arguing that it has a following.