An Interesting Compromise

pen-pen2002 · Post by **pen-pen2002** » Sun May 01, 2005 7:30 pm

The debate over opinion scores has become quite heated lately and I know you are sick of arguing but I think I have had an idea that has not been mentioned yet. I don't claim to have all the answers but I just wanted to throw something out and see what people thought.

The problem: Inflated scores. Most people only use the top 3 numbers when giving opinions.

* Not enough variation to impart meaningful information.
* People using the whole scale are flamed.
* It is impractical to try to change voting patterns in such a large (and often uninformed) population.

The inspiration: I was watching college gymnastics and I realized that scores
never go below 9.5 or so. The scale is the same as ours (1-10) and the averages are similar. There are only two differences: less judges (not very significant), and no limiting factor to score values. Currently if a video is very good and you use the accepted form of scoring you only have two options, 9 or 10.

The idea: open up the scoring at least to the first decimal place.

Pros:
* The system is just taking averages already, possibly very easy to implement.
* New GUI encourages more thought.
* More options encourage lower scores (in gymnastics a 10.0 is a huge deal.)
* More precise scoring, less indecisiveness.
* Would not invalidate previous scores.

Cons:
* A little more time consuming.
* Does not solve fanboy/fangirl all 10 op problem.
* Could decrease number of 8's and 7's given out.

Thoughts?

FurryCurry · Post by **FurryCurry** » Sun May 01, 2005 7:57 pm

Very interesting idea.

I can't think of any way this would make things substantially worse, compared to right now. The fact it keeps the existing scores useful to some degree is a big plus.

I think I like it.

As a side note, there are a couple things that are most likely true about the gymnastics competition you were watching:

Chances are the judges were gymnasts at one time, or have a strong current or former background in the sport, or as a coach or trainer. They know what to look for, more than someone pulled of the street. All the little details of foot placement, blah blah blah.

Also keep in mind that the competing athletes are probably at or near the top of their level of competition. those same judges might well give fives or sixes to performances by gymnasts who weren't good enough to make it to the level of competition you saw.

Malificus · Post by **Malificus** » Sun May 01, 2005 10:10 pm

It would be nice to have. I think it would bring down the perfect tens, replacing them with high nines. not much of a difference, but a definate improvement. To me, 100 choices are better than 10. It would take (almost) all of my perfect tens down. This would open up the ability to give a 9.7, 9.5, 9.6, 10, and 9.8 (or something of that nature), as opposed to all 10s. I say it sounds like a great idea.

And Furry Curry. You are right abbut the gymnastic thing, but this could be used to sort everyone out more accurately. In the big picture, not much would change, but this would help in the greats, or the average AMVs. I don't think it would make a great AMV average or an average on great though.

Still, it's a good idea. I like it too.

Org Profile · Post by **Otohiko** » Sun May 01, 2005 10:19 pm

I'll gladly agree to that. Even being able to give a 7.5, 8.5 and 9.5 would be pretty good - give positive people like myself a bit more elbow room.

But would we still have the old 1-10 marks? In this case, wouldn't that score 'card' get pretty huge?

pen-pen2002 · Post by **pen-pen2002** » Mon May 02, 2005 12:02 am

I don't want to take the gymnastics comparison too far, it was just the inspiration and I'm sure there are better examples. Technically they start from a number based on dificulty and score by deductions, which are often particular set values (eg .5 for falling off).

Otohiko wrote:But would we still have the old 1-10 marks? In this case, wouldn't that score 'card' get pretty huge?

We would have to keep the whole range in order for it to complement the existing system. We would need to design a new interface, probably involving typing in the scores (it would only accept 1-10 of course) hence the slightly lowered ease of use (could be a pro or a con depending.)

dwchang · Post by **dwchang** » Mon May 02, 2005 2:19 am

I still don't see how this would solve the problem of inflated scores which is the main problem no? As you stated, fanboy/girls will still give all 10 reviews, their usefulness will remain low for the community as a whole and "bad" videos will continue to be inflated.

I still advocate a much more thorough usefulness equation and having that dictate the weight of the score. I don't remember all the specifics, but things like how long they have been a member, their top anime shows, possible creator grading opinionators, etc.

I can somewhat see your reasons behind this, but I beive it would just add a layer of complexity (perhaps not that much?) which in reality doesn't help the problem much. Worth it?

Zarxrax · Post by **Zarxrax** » Mon May 02, 2005 10:26 am

I agree with dwchang, I don't see how this would change scores at all.

Look at it this way. Currently people are torn between a 9 and a 10. Some vote 9, some vote 10.

Under the new system, people would be torn between 9, 9.5, and 10, or any decimal value inbetween. I'm willing to bet that any differences in the actual score would be statistically insignificant, because on average, videos would still get the same score.

YouTube · Post by **derobert** » Mon May 02, 2005 1:27 pm

I've got a suggestion. We should have you score between 9.0 and 9.9. Then apply the following foruma:

isa(n) = 10(n-9)+1

Scores using the I.S.A. algorithm will then even be compatible with legacy 1-10 scores.

tuathaanwarrior · Post by **tuathaanwarrior** » Mon May 02, 2005 1:44 pm

derobert wrote:I've got a suggestion. We should have you score between 9.0 and 9.9. Then apply the following foruma:

isa(n) = 10(n-9)+1

Scores using the I.S.A. algorithm will then even be compatible with legacy 1-10 scores.

Are u being sarcastic or are you trying to fight a psychological war with the opinion givers? Cuz that would just be the same as now in all but what you type in rite?

pen-pen2002 · Post by **pen-pen2002** » Mon May 02, 2005 1:49 pm

No, inflated scores are NOT the the problem (IMO).
The problem is that the Standard Deviation is too big:

This is assuming a normal curve when in reality it is probably skewed to the left, but you get the idea. Of course this is also a best case scenario. The current could have been scaled better with some more research (average is more like 8.3) but I spent enough time as it is.

The point is there is not enough variation which in tun gives a high SD which makes the inflated average a problem. Remember 5 is just a number, it's not sacred.

Trying to change the average score given by 300,000 people is nigh on imposible. Weighting is dificult at best and might not help the problem. For example I am a relitivly longtime member, have a usefulness score over 1,000, and don't have DBZ on my fav anime list yet my average score is about the same as the global.

Average is irrelevent. Even if you could change the average you would invalidate all the scores we have now.

Really though the two ideas are not mutually exclusive, they just address difrent problems. Mine focues on the lack of variation, weighting adresses the problem of large numbers of fans drownding out the experts opinions. That's why Olympic diving, for example, is judged by a panel, not the audience.

Why do you hate all ten ops? Because they contain no information.

Malificus wrote:It would take (almost) all of my perfect tens down. This would open up the ability to give a 9.7, 9.5, 9.6, 10, and 9.8 (or something of that nature), as opposed to all 10s.

Bingo. Help make tens meaningful again.

This is not a garanteed solution, however, I belive it is worth implementing just for the extra feedback value and it has the potential to really change things for the better.