Opinion average adjustment

Keeper of Hellfire · Post by **Keeper of Hellfire** » Sat Aug 06, 2005 4:30 am

Today I have downloaded a video from the recommended videos forum out of curiosity, because the poster claimed it to be "the best amv i have ever seen". Every time I read such a statements I get my doubts. The same poster left an opinion with straight 10s. Presumablyy it is the only AMV he as ever seen, or the other have to be really bad, because only from reading the specs I knew neither video quality nor audio quality deserve a 10. I watched it, and right, it was full of compression artefacts.

While opinions are meant to be as help for the creator to improve and not as decision help for the audience, they still are used that way because the averages are visible. In many contests it is usual to calculate an average after scratching the best and the worst result. My proposal is a little bit different: take straight 10s and straight 1s (even if thy mostley are removed on request of the creator) out of the calculation of the average. That way reviewers are forced to think a little bit about at which point a video isn't perfect.

bum · Post by **bum** » Sat Aug 06, 2005 8:10 am

Nice idea, though theirs probably the occasional vid that deserves a perfect 10. Personaly I wouldnt mind if the scoring system was removed alltogeather.

Arigatomina · Post by **Arigatomina** » Sat Aug 06, 2005 1:14 pm

We already drop the high and low reviews when calculating the scores for the top 10% list. So the first all-10s review is automatically ignored, along with the first all-7s review (since a review with all 1s would be considered a flame and deleted, the 'low' would probably be in the 5-7 range).

If we ignore all reviews giving 'all 10s' and 'all 7s' then what? Do we start ignoring the 'all 9s' that people use when they aren't allowed to mark 10? If someone thinks a video is perfect, he's going to give the closest he can to a perfect score. If you don't let him put perfect 10s, he'll put perfect 9s (the next best thing, and the 'new' perfect after the 10s are removed).

You can keep moving the litter box closer to where the cat is peeing, but until you teach him to pee *in* the box, your floor will still stink.

Keeper of Hellfire · Post by **Keeper of Hellfire** » Sat Aug 06, 2005 2:33 pm

Arigatomina wrote:Do we start ignoring the 'all 9s' that people use when they aren't allowed to mark 10?

I hope it isn't an intentional misinterpretation of what I said. I meant only to leave out ops which have 10's in all categories. I don't want to forbid 10's, the only thing I say is that no vid is that perfect that all 10's are justified. And maybe it is a way to teach to take a better look at the vid if you have to give at least one 9 in one category if you want to have your op in calculation. Noone stops you to give all 10's if you want.

Kalium · Post by **Kalium** » Sat Aug 06, 2005 2:59 pm

Keeper of Hellfire wrote:
Arigatomina wrote:Do we start ignoring the 'all 9s' that people use when they aren't allowed to mark 10?
I hope it isn't an intentional misinterpretation of what I said. I meant only to leave out ops which have 10's in all categories.

Ari has a valid point you seem to have missed, though. As soon as you do that, people will find a way around it to accomplish the same thing. The end result is the equivalent of an arms race, where the best you can hope to do is keep things from going badly for you.

Solaria735 · Post by **Solaria735** » Sat Aug 06, 2005 3:40 pm

Exactly, people will start giving videos all 10s except for one category, in which they'll give a nine. And, as bum said, there MAY be the occasional video that deserves all 10s. I think no matter what the system is, people will find a way around it. :-\

Keeper of Hellfire · Post by **Keeper of Hellfire** » Sat Aug 06, 2005 4:19 pm

I didn't miss the point that it still can be abused. But you can set some hurdles.

It seems to me that a point I implied got missed. If the op system is intended to be a help for the creators to improve, how helpfull are straight 10's? If you look some pages back, you'll find a topic from me in which I'd suggested to have all op's public. That way you could easy figure out if the high score of a vid only result from straight 10's of fans of the anime, the music, the creator and so on. The answer was "No, because op's are for the creators and not for the ones who want to download, to not influence the reviewers." But if so, why is it used as ranking sytem too? Even seeing the averages from former op's influences a reviewer. And if the opinions, including the score averages, are visible only to the creator, the abuse of the system immediatly will stop, because it brings no advantage any longer.

The op system as is is crap. Either there should be higher hurdles to abuse it (you can't never stop the abuse, only decrease it), or stop using it as ranking system at all. Isn't the star ranking enough?

Kalium · Post by **Kalium** » Sat Aug 06, 2005 5:58 pm

Thing is, when said hurdles make it more difficult for legitimate ops to be left, then they have not only failed but backfired.

Besides, if you would read more carefully, you would know that the top 10% lists are calculated with the top and bottom 10% of the opinions on each video removed. It's posted on the bottom of the page.

Keeper of Hellfire · Post by **Keeper of Hellfire** » Sun Aug 07, 2005 3:38 am

Kalium wrote:Besides, if you would read more carefully, you would know that the top 10% lists are calculated with the top and bottom 10% of the opinions on each video removed.

I read this. But I didn't understand which 10% are left out, because the personal average is calculated against the orgs average. Is it for the orgs average, the personal average or both? I don't know why, but I assumed it was for the orgs average.

Besides, the more I think about it, the more I get convinced that the op system shouldn't be used as ranking sytem. It's too vulnerable for abuse. It leaves too many vid's out, because they have less than 7 or no ops at all.

Kalium · Post by **Kalium** » Sun Aug 07, 2005 10:24 am

Keeper of Hellfire wrote:Besides, the more I think about it, the more I get convinced that the op system shouldn't be used as ranking sytem. It's too vulnerable for abuse. It leaves too many vid's out, because they have less than 7 or no ops at all.

Any sort of rating system is subject to abuse. It comes with the territory.