The Reviewer's Room

A place to discuss consumer review writing aimed at the UK-based opinion sites such as Dooyoo
 
HomeHome  CalendarCalendar  FAQFAQ  SearchSearch  UsergroupsUsergroups  RegisterRegister  Log in  

Share | 
 

 Is peer review really the best way forward?

View previous topic View next topic Go down 
AuthorMessage
Bluebloodedscouser

avatar

Posts : 179
Join date : 2009-08-05

PostSubject: Is peer review really the best way forward?   Thu Aug 06, 2009 1:23 pm

With the blatant overrating that you can see on dooyoo and pretty much all sites is peer review really the best thing for a consumer? Helium.com has an anonymous rating system which I favour but then the rankings on that are as random as anywhere else. Would it be a better idea just to let consumers read reviews at random or in newest order with members paid a set value for each review?
Back to top Go down
View user profile
mattygroves

avatar

Posts : 276
Join date : 2009-08-05
Age : 50
Location : SW London

PostSubject: Re: Is peer review really the best way forward?   Thu Aug 06, 2009 1:33 pm

I see merit in that, but can also see problems. There is already so much dross on all the sites, that the consumer needs a way to separate the wheat from the chaff. Now, you could hire staff (or con guides) into doing it, but at least with it, there is SOME criteria the product comparing consumer can use.

I'm not sure, at the moment, what the solution is - I surely like the feedback of commenting and interacting with some of the folk who pop by my reviews.

I think it partly matters whether a review site wants to have a community aspects (which relies on positive feedback - and by that I don't mean that all feedback should be positive, but that one sort of feedback encourages more), or whether it is just a business model.

Beer in the Evening (www.beerintheevening.com) has no peer review, and you cannot edit reviews once written (though you can add multiple reviews of a single pub), and tends to mostly attract two line reviews. Having said that, when you're looking simply for a boozer near a venue you are visiting, that can be enough. But having said THAT, it tends to attract pub landlords to rate their own pubs...

dunno, really!

_________________
I refuse to engage in a battle of wits with an unarmed opponent.

The right to be heard does not automatically include the right to be taken seriously.
- Hubert H. Humphrey
Back to top Go down
View user profile
helencb

avatar

Posts : 292
Join date : 2009-08-04
Age : 49
Location : Near the Trent, East Bridgford, Notts

PostSubject: Re: Is peer review really the best way forward?   Thu Aug 06, 2009 1:54 pm

Given the number of sites out there that do NOT do it, and sites that people pay attention to - e.g. tripadvisor, amazon, etc, I think there is a good argument not to do it.

I think, then, that payment could be made on a share basis, on non member reads somehow. I realise that might have some problems which would need some thought, e.g. I could click my review every day to boost income, but I think that might be preferable than say a higher base rate than dooyoo, which, while it has attracted some volume, has also attracted a lot of people who purely want to money make, (nothing wrong with that per se) but it is the type of reviews being added, which are not consumer centric that then becomes an issue.

I dont think the peer rates really help the consumer. Arguably on ciao they do as they rank them somehow, but dooyoo defaults to the latest, and I think you would really need to be looking, more likely to read one or two reviews and move on.
Back to top Go down
View user profile
Cat19

avatar

Posts : 367
Join date : 2009-08-05
Age : 47
Location : Hertfordshire

PostSubject: Re: Is peer review really the best way forward?   Fri Aug 07, 2009 1:48 am

In two minds about this one. Sometimes I go through phases of thinking there is so much trouble on dooyoo due to the rating and commenting that it would be nice to make this anonymousr do away with it. Then we could all have a nice quiet peaceful life like we (some travel writers) have on IGOUGO for instance. On the other hand, it would really change the nature of dooyoo, the community aspects would just disappear and ultimately I don't think that is really what I would want.

From the non-consumer point of view, I wonder if it would make much difference at all. The ratings from member perhaps sort out the really bad reviews but probably not the mediocre versus the good. Some of the current comment threads must leave non consumers bemused.
Back to top Go down
View user profile
Dusky

avatar

Posts : 20
Join date : 2009-08-07
Location : duskland

PostSubject: Re: Is peer review really the best way forward?   Fri Aug 07, 2009 12:13 pm

I do really like the model igoyougo.com uses. There is enough community feel with the cheers and friends lists without all the downsides of Dooyoo. Having the best of does sort of sort the wheat from the chaff If the comments section was kept there would still be a community feel of some sort.
Back to top Go down
View user profile
marymoose99

avatar

Posts : 148
Join date : 2009-08-07
Age : 36
Location : Workington, Cumbria

PostSubject: Re: Is peer review really the best way forward?   Fri Aug 07, 2009 1:50 pm

I also like Igougo, but that's really because I just don't have time for another review site on top of Ciao and Dooyoo which requires reading and rating other people's.

Personally I wouldn't like to see dooyoo change the way rating is done. However, I would like to see members starting to be a bit more honest in their ratings. I think things are getting worse, with really established members who are good writers giving out VH ratings to barely useful reviews.

I think to some extent things are even worse on Ciao with the Exceptional ratings, and the worst thing is where you see people actually saying in their "About Me" that they are thankful for their Es and now need to give Es back to those who have given them Es. On a side note it does make me chuckle that Ciao is probably one of the few places where people say thank you for Es!!!

I would hate dooyoo to go down the Helium route, but that's probably because I just couldn't get to grips with Helium and being made to rate reviews I really didn't want to read in the first place - I want to choose what i want to read and when I want to read it! I do like to see who has read my reviews so that I can reciprocate ratings.
Back to top Go down
View user profile http://www.sis-cumbria.co.uk
Dusky

avatar

Posts : 20
Join date : 2009-08-07
Location : duskland

PostSubject: Re: Is peer review really the best way forward?   Sat Aug 08, 2009 3:00 am

Do you think if the previous ratings were hidden until you actually rated it (like in online polls)might help over rating as you do have to use your own powers of judgment and less of a fear of not beieng the odd one out.

The other way it could be done is not showing who the author is in the new in list. If you are genuinly interested in the subject of the review should it matter who wrote it? Then again that may lose the community spirit but if you could still click through to your circle of trust you could still read favourite authors and clicking cartles would become more obvious.
Back to top Go down
View user profile
Cat19

avatar

Posts : 367
Join date : 2009-08-05
Age : 47
Location : Hertfordshire

PostSubject: Re: Is peer review really the best way forward?   Sat Aug 08, 2009 5:11 am

Dusky wrote:
Do you think if the previous ratings were hidden until you actually rated it (like in online polls)might help over rating as you do have to use your own powers of judgment and less of a fear of not beieng the odd one out.

Would only work towards honest ratings if you could not change the rating afterwards and I think people do have genuine reasons for changing a rating in many circumstances. Hit the wrong button or review is updated being the two common ones.

Dusky wrote:
The other way it could be done is not showing who the author is in the new in list. If you are genuinly interested in the subject of the review should it matter who wrote it? Then again that may lose the community spirit but if you could still click through to your circle of trust you could still read favourite authors and clicking cartles would become more obvious.

Not showing the author is interesting idea though. Or allowing the author to see who has rated, to see what the ratings are (i.e. 20 Vu's, 3 U's etc) but not matching up who rated what. This might retain some of the communiy spirit but avoid the bickering over rates. Of course some people would then just sit and watch their reviews all day to try and work out who rated what. Laughing
Back to top Go down
View user profile
Camp Freddy

avatar

Posts : 235
Join date : 2009-08-08

PostSubject: Re: Is peer review really the best way forward?   Sat Aug 08, 2009 10:53 am

I've always advocated a 1 - 10 rating system. I think it gives a lot more flexibility than just having four options, plus taking the descriptive element out of it ("useful"), then means people should be more inclined to vary in their rating. If you added an anonymous rating aspect to this, I think this is a system far better than Dooyoo currently has.

Ultimately, each review would be given a numeric rating out of ten, with ten being a perfect review and spotting people constantly giving out 9s or 10s should be a lot more obvious to the Dooyoo team.
Back to top Go down
View user profile
helencb

avatar

Posts : 292
Join date : 2009-08-04
Age : 49
Location : Near the Trent, East Bridgford, Notts

PostSubject: Re: Is peer review really the best way forward?   Sat Aug 08, 2009 12:22 pm

Trivago has that...someone gave me an 8 the other day...GRRR
affraid

only kidding
Back to top Go down
View user profile
marymoose99

avatar

Posts : 148
Join date : 2009-08-07
Age : 36
Location : Workington, Cumbria

PostSubject: Re: Is peer review really the best way forward?   Sat Aug 08, 2009 1:27 pm

A 1-10 rating system could be interesting, but I think how I rated would possibly change more depending on my mood, and how annoyed I got at people writing it's instead of its!!!
Back to top Go down
View user profile http://www.sis-cumbria.co.uk
Camp Freddy

avatar

Posts : 235
Join date : 2009-08-08

PostSubject: Re: Is peer review really the best way forward?   Sat Aug 08, 2009 1:52 pm

That's no different than how you current rate though Mary, is it? NO-one's 100% consistent in that vein.

Getting rid of the labels is what would do the most good. People are personally affronted that their reviews aren't seen as "VU" whereas I believe people would be more open to getting a score of, say, 7 out of 10. (Which probably equates roughly to a Useful).
Back to top Go down
View user profile
marymoose99

avatar

Posts : 148
Join date : 2009-08-07
Age : 36
Location : Workington, Cumbria

PostSubject: Re: Is peer review really the best way forward?   Sun Aug 09, 2009 4:47 am

I see what you mean...but I don't know, it might make me have to think more (that's probably not a bad thing)...at the end of the day though I'm sure I'd get used to any changes made!
Back to top Go down
View user profile http://www.sis-cumbria.co.uk
jojopillo

avatar

Posts : 7
Join date : 2009-08-11

PostSubject: Re: Is peer review really the best way forward?   Tue Aug 11, 2009 6:07 pm

Bluebloodedscouser wrote:
With the blatant overrating that you can see on dooyoo and pretty much all sites is peer review really the best thing for a consumer? Helium.com has an anonymous rating system which I favour but then the rankings on that are as random as anywhere else. Would it be a better idea just to let consumers read reviews at random or in newest order with members paid a set value for each review?

When you go on to someones profile and you see how many VU's U's SU's and NU's they have, it could be shown how many VU's, U's etc they have given. Maybe this way they would think before giving VU's on everything.
Back to top Go down
View user profile
Bluebloodedscouser

avatar

Posts : 179
Join date : 2009-08-05

PostSubject: Re: Is peer review really the best way forward?   Tue Aug 11, 2009 11:25 pm

jojopillo wrote:
Bluebloodedscouser wrote:
With the blatant overrating that you can see on dooyoo and pretty much all sites is peer review really the best thing for a consumer? Helium.com has an anonymous rating system which I favour but then the rankings on that are as random as anywhere else. Would it be a better idea just to let consumers read reviews at random or in newest order with members paid a set value for each review?

When you go on to someones profile and you see how many VU's U's SU's and NU's they have, it could be shown how many VU's, U's etc they have given. Maybe this way they would think before giving VU's on everything.

An interesting idea but I can see the usual problem there. Only those of us who navel gaze will pay any attention to these stats. The majority would remain oblivious to members rating behaviour.
This might cause issues in itself too. I have challenged blatant overrating before and it often causes argument.
Back to top Go down
View user profile
cmh4135



Posts : 177
Join date : 2009-08-09

PostSubject: Re: Is peer review really the best way forward?   Wed Aug 12, 2009 1:41 am

jojopillo wrote:
When you go on to someones profile and you see how many VU's U's SU's and NU's they have, it could be shown how many VU's, U's etc they have given. Maybe this way they would think before giving VU's on everything.

Hi Jojo!

I've often thought that I'd like a snapshot of how people rate (and for others to be able to see how I rate as I'm not as harsh as I think people think I am!! Razz ). I do think, however, that this might cause even more of a ratings issue. If X decides that they want lots of rates and all VUs then they concentrate on finding those members who return rate and rate all VU... a bit like the message that went out to avoid guides cos if you stay under their radar then your less likely to pick up anything other than a VU.

I think people stress about rates too much and it's only our own curiosity and egos that make us this way. Ask anyone whether it REALLY matters and I'd guess the vast majority would have to say no.

I do, however, think that there is some consumer merit in peer review. If I research a product on Amazon or Trip advisor then I'm most likely to ignore the top and bottom reviews (in terms of numbers of product stars given) and read where the bulk of ratings lie. I will then glance at the top and bottom ones just to highlight any major issues. However. If I'm looking at a particularly popular product it can be a faff wading through nonsense reviews or reviews that really say nothing. What I'd actually like to be able to do on an opinion site is search according to product rating AND perceived usefulness to the consumer. That way I'd lessen the need to read 101 ops that were all similar in search of that one bit of info that I was really looking for.
Back to top Go down
View user profile
Camp Freddy

avatar

Posts : 235
Join date : 2009-08-08

PostSubject: Re: Is peer review really the best way forward?   Wed Aug 12, 2009 6:39 am

Does anyone remember when Ciao published people's rating habits? It was moider!
Back to top Go down
View user profile
cmh4135



Posts : 177
Join date : 2009-08-09

PostSubject: Re: Is peer review really the best way forward?   Wed Aug 12, 2009 6:45 am

I don't recall (yet I was more active there than dooyoo for years)
Back to top Go down
View user profile
Camp Freddy

avatar

Posts : 235
Join date : 2009-08-08

PostSubject: Re: Is peer review really the best way forward?   Wed Aug 12, 2009 6:50 am

There was a mad rush of people running round, seeing what people's rating percentages were, then it was all taken off-site onto messageboards like Chatterweb or Chuckleweb and people compared 'scores'. Those were the polite people... having a laugh and poking fun at each other. The Queen of Ciao's inner circle went into overtime with snidey comments and stuff. Unreal.
Back to top Go down
View user profile
marymoose99

avatar

Posts : 148
Join date : 2009-08-07
Age : 36
Location : Workington, Cumbria

PostSubject: Re: Is peer review really the best way forward?   Wed Aug 12, 2009 6:56 am

But you can still see people's statistics on Ciao if you go into their statistics....
Back to top Go down
View user profile http://www.sis-cumbria.co.uk
Camp Freddy

avatar

Posts : 235
Join date : 2009-08-08

PostSubject: Re: Is peer review really the best way forward?   Wed Aug 12, 2009 8:45 am

Yup, but there was a time when you couldn't see people's reviewing statistics at all and then, all of a sudden, they announceed it one day and there was this stampede.
Back to top Go down
View user profile
Sonia



Posts : 130
Join date : 2009-08-06
Age : 33
Location : London

PostSubject: Re: Is peer review really the best way forward?   Wed Aug 12, 2009 9:08 am

I would like to see how other people rate (although you can tell the people who over rate everyone because they rate even the worst reviews as VU) but I wouldn't want people to see how I rate. I think some people would stop rating me if they could see how many NU's and SU's I give out (is it just me or is the quality of reviews this week and last week a lot worse than usual?).
Back to top Go down
View user profile
Camp Freddy

avatar

Posts : 235
Join date : 2009-08-08

PostSubject: Re: Is peer review really the best way forward?   Wed Aug 12, 2009 10:01 am

There's long been a correlation, I think, between how you're perceived as a rater and the number of reads/rates you get in return. So-called "harsh" raters rarely perform as well as those that are more generous - based on what I've seen on Ciao where rate activity is available.
Back to top Go down
View user profile
helencb

avatar

Posts : 292
Join date : 2009-08-04
Age : 49
Location : Near the Trent, East Bridgford, Notts

PostSubject: Re: Is peer review really the best way forward?   Wed Aug 12, 2009 11:46 am

interesting point, I remember the furore at Ciao too. It is amazing how we all live with things eventually though.

I dont tend to notice the reviewer when rating, as a general rule, so I dont think it would matter to me one way or the other.
Back to top Go down
View user profile
Sponsored content




PostSubject: Re: Is peer review really the best way forward?   

Back to top Go down
 
Is peer review really the best way forward?
View previous topic View next topic Back to top 
Page 1 of 1
 Similar topics
-
» Follow up review of the Gold Lion KT66's..... :}
» 3-D TARDIS Model Kit book review wanted
» Third Doctor & Sea Devil Review
» Review on set 8019 Republic attack shuttle
» Review: 8957 Mine Mech

Permissions in this forum:You cannot reply to topics in this forum
The Reviewer's Room :: Review Writing :: Dooyoo.co.uk-
Jump to: