Hall of Fame or Amazon pop chart history?
Amazon's official announcement
Hello Top Reviewers!
We'd like to honor our superstar reviewers with a new Hall of Fame. Many of you have been writing for years and have become pillars of the reviewing community. We know your names. Customers know your names. You deserve a bit of glory for your long term dedication, which has amounted to an invaluable contribution to the Amazon community.
We are creating a new Hall of Fame Reviewer page that will acknowledge the multi-year accomplishments of our elite reviewers. Inducted members will receive a permanent badge that will show next to each one of their reviews. To kick off the Hall of Fame, we scanned through our archives starting from the year 2000 and will induct top reviewers from the last 11 years.
Here's how it will work:
- Any reviewer who rises to the top 10 rank on Amazon, even for a day, will be recognized as a Hall of Fame reviewer and immediately inducted.
- We specially inducted the top 100 reviewers from the Classic Ranking as of today. (If you think you were in the top 100 and we missed you, please let us know.)
- We will highlight the number of years each reviewer has achieved the Hall of Fame. We are awed by the reviewers who made it to the top ten year after year.
- Since the Hall of Fame is a much stronger representation of our oldest and best reviewers, it will replace the Classic Reviewer Ranking.
- There are no changes to the New Reviewer Ranking - it will simply be called the Top Reviewer Ranking from now on.
It's a big change, and we think it's a good one. As always, we want to hear what you think too.
In the UK, the announcement was edited to say that only the top 10 had been specially inducted.
Background
At first glance, the so-called Hall of Fame might seem fair enough, but in reality it is another kick in the teeth for those long-time reviewers who were with Amazon in the early part of the last decade, and sometimes even earlier. For the most part, these reviewers focused on books, music, movies and games - the things that made Amazon a household name in the first place, allowing it to expand into other areas.
The current ranking system skewed things in favor of gadgets and household stuff at the expense of books, music, movies and games. As of now, there looks superficially to be a diverse range of reviewers in the top 100, with all types of products represented. But I once remember somebody saying, in a completely different context, that it's not where you are at, but where you are heading, that matters most. I recognized that the second ranking system was heading for stagnation as far back as 2003, and I remember being ridiculed for suggesting it, but I wasn't even the first to notice it. It isn't difficult to see that the current ranking system is creating a different type of stagnation in which reviewers of books, music, movies and games are gradually being forced out of the top 100. Maybe it will take a few years for it to become obvious to everybody, but it's obvious to me. The reason for this squeeze is the heavy penalties for negative votes. In general, gadgets and household stuff don't attract such penalties to anything like the extent that books, music, movies and games do. No I don't want negs abolished, just to have a fairer balance between YES and NO. Abolition would be worse than what we have now.
So if ranking is a part of people's motivation, the current ranking system undermines that motivation for reviewers of books, music, movies and games. Fortunately, I realized a long time ago that rankings (on both systems) were a load of bull, Yes, there was a time when I was competitive in the rankings, but that all ended years ago. My motivation these days comes from the feedback I get in various forms, including comments on the review that sent my ranking into freefall. I don't care much for either ranking system and wasn't sorry on a personal level to see the second system abolished, but I empathize with some of my friends who placed more value on it.
If the Hall of Fame was an attempt to appease the old-timers, well, it might appease some a little and others a lot, but I'm not impressed.
Analysis
After going to the ranking table page, and clicking on the hall of fame tab in the location where the tab for the second rankings used to be, I saw the hall of fame page. It probably didn't include everybody who should have been there, but I saw enough to get a feel for what it is really about. Combining Amazon's announcement with what I saw for myself and what other people were saying about it, I quickly realized that it isn't a hall of fame in the true sense at all. Anybody who is aware of halls of fame in music, sport, showbiz or whatever, knows that these reward people who have achieved great things, usually over a significant period of time and often after their careers are over. I was not surprised that Amazon stretched the definition somewhat, devaluing the very idea of a hall of fame, but they have gone way beyond that. Remember this line from the official announcement.
Any reviewer who rises to the top 10 rank on Amazon, even for a day, will be recognized as a Hall of Fame reviewer and immediately inducted.
As I pondered that sentence, it occurred to me that a rock band wouldn't automatically get inducted into the Rock and Roll hall of fame because of one top ten hit. At that point, I realized what Amazon had created. It's not a hall of fame, but a pop chart. Some Rock and Roll Hall of Fame inductees have lots of top ten hits, and it may be that some others have got in without having any top ten hits, if they have done other things to deserve the accolade. Pop charts never tell the whole story, nor do reviewer rankings come to that. On the other hand, some people have had an abundance of hits and never been inducted. Now, I like the idea of an Amazon reviewers' pop chart, but it really shouldn't be called a hall of fame. A hall of fame should be reserved for special achievements. They have introduced a #1 Hall of Fame reviewer badge to distinguish those few reviewers who made it to number one at some point, but it's not exactly distinctive, is it?
So all the bull about superstar reviewers, long term dedication and other flattering words counts for nothing, because somebody can get into the top ten for one week, disappear never to be seen again and still be inducted into the Hall of Fame. Against that, somebody who has spent several years in the top 100, perhaps even the top 20, without quite making it to the top 10, gets nothing. If they survived in the top 100 on the second system till the end, they do get a badge, but I know good reviewers who failed to do that. In any case, the top 100 clause only applies in the USA. My several years in the USA top 20 (and several before that in the top 100) counts for less than somebody who sneaks into the current top 10 for just one week. So much for this then.
We are creating a new Hall of Fame Reviewer page that will acknowledge the multi-year accomplishments of our elite reviewers.
In my case, my multi-year contributions have been recognized in the UK, but that's not the point. Plenty of others haven't been recognized that deserved it more than somebody who makes the current top 10 for one week. If Amazon had really wanted to reward multi-year contributions, they would have created an algorithm based on rankings over a period of time - say 10 points for top 10, 9 points for ranks 11-20 and so on down to 1 point for ranks 91-100, then added up the points and awarded hall of fame status to those with the highest points totals. Anybody else who reaches that points total in future would then get in the hall of fame. The formula I gave just came off the top of my head and there are plenty of other ways of achieving the sort of result I have in mind, which would be a true hall of fame.
Somebody suggested that perhaps Amazon didn't recognize all the older reviewers who deserved to be in the hall of fame because they wanted to limit the numbers. That might be a fair argument had they provided comparable limits going forwards, but they didn't. A place in the top ten on the second system was much harder to achieve than the same place is on the current system. If they had said that there would be a lengthy minimum qualifying period (say six months in the top ten), that would at least show that Amazon have some understanding of the concept of a hall of fame, but it is still easier to stay in top ten for six months on the current system than it was to make the top ten on the second system, except perhaps in its very early days, which were before my time as a serious reviewer. (I posted my first review in April 2000, but only took reviewing seriously in June 2002.)
It would still be entirely valid to keep the pop chart that we now have, but call it something like All top 10 reviewers ever. I like that idea, but it isn't a hall of fame. As with the Rock and Roll hall of fame, it would be possible to get into the hall of fame I suggested without ever getting into the top 10, but obviously getting into the top 10 makes a hall of fame entry more likely.
There's another line in the announcement that illustrates how out of touch Amazon are. Remember this line.
Since the Hall of Fame is a much stronger representation of our oldest and best reviewers, it will replace the Classic Reviewer Ranking.
Note the reference to our oldest and best reviewers. Amazon may think the hall of fame more strongly represents them than the second ranking system. Those who fit the description oldest and best reviewer and who also think the hall of fame more strongly represents them than the second ranking system would be in a small minority.
While I long expected the second ranking system to go and didn't mind that, the one thing it did well was represent the oldest reviewers. Unfortunately, because it allowed inactive reviewers to keep their ranking points, it represented them all too well.
Ideas to consider
It would be possible to represent older reviewers by creating a version of the current ranking system (or any future system that might replace it) with the aging element removed. That system would run alongside the current system (or any future system that might replace it) and would be called Lifetime rankings or some such. Because the basic algorithm would be the same for both systems, differing only in the presence or absence of the aging factor, you wouldn't have the bizarre situation of somebody being among the best on one system and among the worst on the other. Whether somebody has a poor helpfulness ratio or a good one, taking away the aging factor won't alter that. People can accept that a top 100 reviewer on a lifetime ranking might only rank 25,000 or whatever if they are inactive, but for a top 100 reviewer on one system to be ranked 8 million on another is nonsense.
As Amazon have recorded people's rankings over a number of years (something I didn't know about until the Hall of Fame was launched), it should be possible for Amazon to show Peak position on profile pages under or alongside current rankings. That peak position could be for either ranking system. It happens that my USA peak of #8 was reached on the current system; my peak in the second system was in the teens. I never expect to come close to those numbers again, because gadget reviewers are gradually taking over the top 100 (and that's also happening in the UK) and I couldn't compete with them even I wanted to. Still, it would be a nice gesture on Amazon's part if peak positions were shown on profile pages.
If Amazon were to consider both the lifetime rankings and the peak position, the peak position for the second system need only be reflected in the Lifetime system.
The other issue I've seen raised is the paradox whereby reviewers are considered good enough for the Hall of Fame but not for Vine™ membership. Although I've never been impressed by Vine™, I am sure that given a choice, most reviewers would rather be offered Vine™ membership than induction into the Hall of Fame. Some older ex-reviewers who have quit might even be tempted back to reviewing with such an offer.
Can there be a fair ranking system?
I recognize that the second system was skewed, because it favored those reviewers who got in early. If you're going to skew the system, you should favor active reviewers and the current system does that. Whether it does so to excess (and it might do, though I am not sure about that) is an interesting point. However, skewing the system to favor reviewers of gadgets and household stuff at the expense of book, music, movie and game reviewers is not acceptable, except of course to reviewers who benefit.
Of course it's Amazon's system and they can and do change it whenever it suits them. No ranking system will ever please everybody and I don't see how it could. Most people judge a ranking system with a certain amount of self-interest, I try to be as dispassionate as possible, on the basis that if I write reviews that customers find helpful, and plenty of them, I'll have a ranking reflecting that. I'm not bothered about free books from authors and publishers and in any case don't like receiving e-mails from those who haven't looked at my other reviews to see if I'm the sort of person who might like their book. I don't particularly care for the media attention that goes with being #1 and am happy to no longer be #1. Debbie Jones may not have wanted to be #1, but after initially deciding to take a long break from reviewing, she must have decided that a short break was enough. My days as UK #1 reviewer may not be over for good, because either a fourth ranking system or a lack of activity by other reviewers could propel me back to the top. So if any high ranking reviewer can be dispassionate about the fairness or otherwise of reviewer ranking systems, I can.
