Ratings guidance from Dooyoo Towers

Dooyoo has posted an essay, near enough, on ratings (even using the dog/hotel example):

http://members.dooyoo.co.uk/community/

Nice to get some guidance - particularly in line with what many of us have been banging on about for some time.

At least they mentioned irrelevant text, that might stop people using Dooyoo as a showcase for their writing skills and get them to write reviews instead.

That's your interpretation of 'irrelevant text' - but it's not necessarily what they mean or the only interpretation.

Irrelevant text could include ingredient listings, detailed info about the design of the packaging (I believe someone once described the bar code), info quoted directly from the website etc., the entire history of the Kit Kat in sufficient detail to take it as a specialist subject on mastermind.

I am quite pleased to see that dooyoo have commented on this topic, I was beginning to thing that accurate rating / over rating was quite a low priority.

So 95% of ratings VU, I would have guessed high, but not that high.

Remind me - did it say that 95% of the ratings were VU or 95% of the reviews got a VU grading. The two wouldn't necessarily be the same thing.

Oh I will have to go and re read, that seems slightly more likely.

Quote :

The fact is that the dooyooCommunity rates well over 95% of the reviews posted as Very Useful.

Still could mean either - not that it's important either way. It's still a shocking reflection that some people must be reading with their eyes closed.

Essays are good. Actually taking action on people would be better.

I know you can't get 'em all, but if someone's reading everything from the new in list and rating 95% of it as VU, then take their miles off of them FFS.

Dooyoo need to do more than that...supporting people who do rate honestly would be a good start.

Some random thoughts from Four-Eyed Monkey Land...

Although this is certainly welcome, I can't help feeling it's perhaps a case of bolting the stable door after the horse has bolted. I'm not sure what the solution is, but maybe it's time for a complete overhaul of the rating system. Off the top of my head, there's a few potential options I can think of: moving to a five/ten star rating system, restricting how many VU people can give in a single day, not allowing people to see how others have rated the review (so they can't be influenced) or even doing away with the rating system completely and just recording how many times a review has been read (with appropriate safeguards built in to guard against speed-clicking, of course!)

The trouble is, all of these alternatives are open to abuse in one form or another and either simply duplicate the problem in a new format or bring in whole new ones.

The only way to reinforce the message is as Camp Freddy says to actively take action. We've all seen examples of badly spelled, uninformative, just over minimum word reviews with loads of VU ratings - the people who gave those ratings need to be challenged to justify them and warned on future conduct if they can't do so, with subsequent offences resulting in loss of Miles/bans, whatever. The trouble is I don't think Dooyoo have the staff to be able to comb through the reviews in this level of detail.

Done in the right way, I don't think most members would take offence. I was once challenged by a Guide who asked me whether I honestly thought a review I had rated as VU deserved that. I explained my reasons for the rating, they accepted and were happy with them, end of story! I certainly didn't take offence at being challenged and it did cause me to go back and reflect on the rating given (even if I didn't change it).

As things stand, the ratings are virtually worthless as a guide to how good a review is/isn't, so doing away with them completely would scarcely impact on the value of the site.

I also find the rating thing a rather peculiar affair.....i.e. why do thousands of us read about things that we are largely never interested in buying.....all for a few pence....I have long thought it is time for a bit of a rethink as well.

My personal favourite is, as mentioned above albeit in slightly different form, a 1 - 10 numbered system. I think once labels are taken off of reviews and reviews are scored numerically, a lot of the stigma of lower ratings goes away.

People would be more inclined, I think, to take a 6/10 rather than a Useful, for example. A 10/10 rating would be very rare IMO and anyone consistently rating 9 or 10/10 should be easy to spot on Dooyoo's radar.

Existing reviews can be easily converted as each rating could correspond to 2.5, 5, 7.5 and 10 out of ten. New ratings would then bring this average down (perhaps rounded to 1 decimal place).

If greater emphasis was given to only reading what you were interested in as a consumer, then rates would have more value. Ideally reviewers would still get a reward for older reviews, so raters would have good reason to read older stuff in the database, keeping even older reviews relevant.

I like the idea of rating between 1-10.

A further refinement to the idea: Individual readers could rate reviews, but those individual ratings not be available for anyone to view except the rater. The overall ratings could then be averaged out and that score displayed to all users as Average Rating: 6/10 or whatever.

From the point of view of a writer and not a rater I would really like that...it would be nice to know what people really think of my reviews.

I didn't like the idea of 1 to 10 at first, but it is defintiely growing on me.

I wonder how long it would take for a 9 to be seen as a slight though

I suspect that whatever system they had, some members would find a way to distort it.

SWSt wrote:

A further refinement to the idea: Individual readers could rate reviews, but those individual ratings not be available for anyone to view except the rater. The overall ratings could then be averaged out and that score displayed to all users as Average Rating: 6/10 or whatever.

Perhaps.. and this is only an idea... rates can be made available to Guides so that rating anomalies can be spotted by more than Dooyoo. I know some will continue to whine about guides, but I don't think there is a better system out there - my only qualm being the odd member that simply doesn't fit in with what I think guides should be about. That's another discussion though.

But the hidden rates idea is great. Not only would it hopefully mean better, more accurate rates, but a lot of the guff where witchhunts go on for those that rate honestly would disappear.

koshkha wrote:

I suspect that whatever system they had, some members would find a way to distort it.

More than likely, K. Some systems are harder to game than others though and IMO the current system is easier to play than that suggested above.

The one downside to the hidden rating thing is that the carrot to get people to read i.e. getting reads in return, has gone.

Perhaps in the hidden rates system, the rater can still be seen, just not their rate?