"Average" Scores

[downwithpants]

What score would you give what you would consider an average video?
An 8 because it's the global average?
Or a 5 or 5.5 because it's halfway between 1 and 10?

Here's my 2 cents:
An 'average' video does not deserve anything more than a 5.5. My guess as to why the reason the global average is an 8 is because that many people tend to delete crappy amvs, and keep the good ones they like. When they go to review AMVs, more often than not they review the amvs they really like. Notice how the top commented videos: 'Odorikuruu', 'Engel', 'Euphoria', 'Tainted Donuts', etc. comprise most of the top favorite videos scale. With this large influx of high ratings (probably mostly 9's and 10's), this is going to bring the average way up. There are also people who give undeserving 10's to their best friend's amv's (or even there own), adding to the skew of the score curve. So when some reviewers see the global average, they think to themselves, 'oh i better give this average video a 7 or an 8 because i don't want to make them feel bad by giving a sub-average score.'
I guess what i'm saying is, some people need to be taught statistics. By skewing the score curve so that the average score is in the 8's, it becomes hard to distinguish between the really good videos.

As an analogy: if predominantly smart people took IQ tests, and someone calculated the average IQ score, the average would be above 100. Then the guy convinced the population that the average IQ score was above 100, the average IQ score would continually rise, rendering it useless.

[dwchang]

I see your point, but at the same time, you answered your own question.

As an analogy: if predominantly smart people took IQ tests, and someone calculated the average IQ score, the average would be above 100. Then the guy convinced the population that the average IQ score was above 100, the average IQ score would continually rise, rendering it useless.

The reason the global averages are so high *is* as you said, but at the same time, why is a '7' (C) considered average in school? Because the people at school are within some delta apart from each other in intelligence.

I know it's hard to grasp, but if you're in HS or college, you're already smarter than a very large % of the world's population. Thus, a 70% is average since you *should* be getting that much right compared to your peers.

If the .org's scoring was based on the *entire world*, then yes a '5' IS average, but considering that each of us (even the "bad' editors) is editing in itself means you're above that average. Think about it, the average person in the world does NOT know how to edit correct? Then that would imply that someone who does edit is above the average yes?

That's an argument as to why I place '7' as average. If not that, I also think about the 'curve.' The distribution centers at 7 - 8 and thus to give scores lower is to in essence score below average.

You can't argue with statistic...although you can form your own opinion on how to score which is ultimately what you should do regardless.

So why am I even posting? I just went in a circle.

[downwithpants]

Thanks for your insight.

I understand your line of reasoning, but I find fault with comparing AMV scores with academic grades. It seems that you’re saying an average student deserves a 70 (rather than a 50) because the high school and college population is on average more proficient than the rest of the world in academic subjects. If this were the case, why is it that the national average on the SATs (roughly 500), is halfway between the minimum (200) and maximum scores (800) possible on the test? The average score is roughly the same on the GREs, which have the same minimum and maximum, and I assume that the average GRE taker has had several years of college education, putting him or her ahead of over at least 99% of the population in terms of the skills required for the examination.

This is a guess, but the reason I think that schools give 70s as an average grade is because students traditionally have gotten 70% of the maximum number of points correct on tests, homework, etc. (Hence, the academic grade is a quantitative measure of the students academic ability, or at least they would be if there were no such thing as a teacher’s pet.) Because GPAs are essential indicators of academic ability when applying to college and grad school, schools must be able to show which students are proficient and which students are inept in their subjects. Since students usually get about 70% of the maximum points in a class (from tests, homework, etc.) schools assigned the 70-80% range as the average GPA of 2.0.

However, it is impossible, or at least impractical, to score things like AMVs quantitatively. For example, we can’t say that video X is worse than video Y just because video Y has more digital effects than video X. So there is no reason for us to score AMVs by the same standards that teachers and professors use to grade their students, and no justification to score an ‘average’ video a 7.

Your next point is that if the org’s scoring were based on the “entire world,” then as experienced editors, even our bad videos would be better than the videos that unexperienced editors would make. However, I’m guessing with high confidence that the “entire world” is not going to start making AMVs. Therefore, we don’t need to compare what we now consider average to what we would consider average if the entire world started making videos, because it’s not going to happen. What we are rating is how a video compares to other existing videos available to the public (or at least org members), We are not rating an AMV editor’s ability to make an AMV against the potential ability of the entire world to make AMVs. Again, there is no need score an ‘average’ video above a 5.5.

What I was referring to by my IQ test example is the effect of a misrepresented average (the high IQ scores of the smart people) can have on the statistical average (the average IQ of the global population). I am not saying that we should give ourselves averages of 7s and 8s because we are more capable editors than the rest of the world.

But yeah, I can’t force people to change their rating standards. I’m just voicing what I think is a better way to score videos, and trying to see how people respond.

[Nappy]

This has been discussed alot :l Basicly most use school grading (7 average), some use 5 average. These aren't SAT's, IQ tests or anything. They are fan made videos ppl do in their spare time o_o

I'll give anyone a 5 just for attempting to do one, bonus points for actually doing it well. And I think alot of others don't look so harshly at these scores either. Which is why, even if everyone did school grading, the average is still higher than 7. ppl who give really low scores are taking it to seriously, or have something against you :l

[Arigatomina]

However, it is impossible, or at least impractical, to score things like AMVs quantitatively. For example, we can’t say that video X is worse than video Y just because video Y has more digital effects than video X. So there is no reason for us to score AMVs by the same standards that teachers and professors use to grade their students, and no justification to score an ‘average’ video a 7.

This is why we don't have a rule on how to grade amvs. Effects do not equal a higher score. A vid with no effects can be a 9 video while one laden with effects falls in the 2s for not accomplishing any of its goals. What is the video aiming to do, how far does it go in accomplishing that? We grade *effort* on how much work went into a vid - here a vid with more time-taking effects equal a higher rating. But for the overall score, effects don't matter at all. There is no one thing that makes a vid automatically higher in scores than another vid. You can't say a vid that uses dvd rips is automatically better than one that uses fansubbed footage - there's too many other factors that determine the grade.

With that said, a 50% in school terms is failing. It isn't average because it's an automatic 0. When you look at school grades, 50% = F = 0, 60=1, 70=2, 80=3, 90=4, 100=5 (think of the star scale). Check the numbers, no one who averages below 50 is passing, even if it's only one percent away. If you put that to videos, anything 5 or below is failing, not acheiving what it set out to do. Why is 5 not average, then? Because most amvs manage to do more than they lack. Even a video with horrible footage, bad timing, no effects and shotty audio may not deserve a failing grade if it achieve's the intended story. And the ones that do fail completely (meaning more flaws than positive aspects) are rarely reviewed - as you noted.

But this has been argued before, yes, with statistics and numbers. You can't grade videos the way you grade a test. You can't set down certain things the video has to get correct in order to make the grade. Amvs just don't work like that because we aren't taking tests where the answers are already there. If we were, every video would be the same - the ones that get the answers right would all be 90%, a nice B+, and all have the correct answers - the same answers. Can you go through an art gallery and grade the paintings based on a set grading scale? Saying effects make a video better than one without effects is like saying a painting with more colors and brush strokes is better than one that uses a subtle texture. It just doesn't work that way.

As for the average vid deserving a 5, it depends on what a 5 means. If it's put to academic scales, a 5 means it fails. If you look at it as the midway point between having everything perfect and everything perfectly wrong, that's fine. You'd be saying it deserves a 5 because there is as much wrong as there is good. But you can't say it deserves a 5 because it's 'average' and isn't better than any other video. The only way you can do that is if every other video has as much wrong as there is good. I think the average video has at least a few more things right than the number of things that are wrong. So I can't give a video a 5 just because it doesn't seem to stand out over the average video. Until the average video here is one with as much wrong as there is right, I can't give 5s as the average - to me a 5 will *always* be the halfway point between perfect and absolutely bad (no good at all).

[Scintilla]

I know it's hard to grasp, but if you're in HS or college, you're already smarter than a very large % of the world's population.

No, it just means you're lucky enough to be from a country where education is compulsory, free, and high-quality.

[trythil]

Mathematically, the arithmetic average isn't even a very good tool to figure out what the middle of the pack is.

If you want to find the score that better denotes what a more middle-of-the-pack video is like, ask Phade to add a module to calculate and display the median score. Throw in a standard deviation calculation as well. Both data points would work better as a numerical indication of the "average" video, and are trivial to calculate.

---

Now, for the human factor:

People don't need to be taught statistics, for these reasons:

(1) A re-centering and re-training of the .org opinionators (which is basically what ramming statistics into people's minds would constitute) is not possible. Such a task would assume the existence of absolute judging guidelines, which just don't exist.

(2) With the exception of a few people, most people tend to opine their friends' videos, videos that get hyped up, videos of "known" creators, or, in general, videos that they know they can rely on to satisfy their tastes or social needs. There are very few people who will just download a video just to leave an opinion on it. (I am not one of them.) Unfortunately, such random opinionation is crucial to having a fairly-balanced distribution of scores, which is crucial to have if the average is to be used as a yardstick of performance. We don't have either, and it's likely that we never will.

---

As an aside, something that would be an interesting experiment (assuming more ideal conditions than those that currently exist at the .org) would be to try to "weigh" the opinionator's scores based on what kinds of videos they claim to be their favorite, so that you don't have someone who enjoys predominantly drama videos skewing a hardcore action video too much. Or something like that. However, due to human variation, people abusing the categorization system, and an asston of other factors, that idea will not, and should not, come to fruition.

[koronoru]

The reason the global averages are so high *is* as you said, but at the same time, why is a '7' (C) considered average in school? Because the people at school are within some delta apart from each other in intelligence.

I know it's hard to grasp, but if you're in HS or college, you're already smarter than a very large % of the world's population. Thus, a 70% is average since you *should* be getting that much right compared to your peers.

I think you're confusing percentage grading with what the statisticians call "percentiles". If you are at the 70th percentile, that means you're (smarter, better, whatever) than 70% of the population. But that's not what 70% on a test is supposed to mean. If you score 70% on a test, that only means you got 70% of the questions right, or the quality of your work was 70% of the best possible quality, or similar. The exact definition depends on the test, but it's almost never meant to be "70% is better than 70% of the population" no matter how you define the population. It was never the case that 50 was supposed to be the average, even though it is true that inflation happens (for the reasons you listed and some other reasons as well).

I grade upper-level computer science courses. If I give you 70 out of 100, that means you probably know more about the course material than about 99.9% of the population (in this country) or 99.999% (in the entire world) or maybe about 40% (of your classmates) - it's not actually a great score (the expected class average is in the high seventies), but only a few people qualify to be in the class at all. Those three percentages are percentiles; none of them is equal to your mark of 70.

The org's opinions-scoring system would be most useful if everyone used percentile scoring (adjust things so that the bottom 10% of videos score 0.0 to 1.0, the next 10% score 1.0 to 2.0, and so on) but I don't think there's any hope of getting people to do that. It would mean that half of the videos you review must score less than 5.0 - and in the current environment, a 5.0 is considered to be a pretty strong "this is a bad video" message, reserved for really bad videos. Like it or not, I think we're stuck with something like the school grade curve, and we've even got the same problem that we see in schools - grade inflation. No individual can change things, either; if I started reviewing a lot of videos and giving scores less than 5.0 to exactly half of them, all that'd happen is I'd get a reputation as being that guy who shits on everyone's work.

[Farlo]

i think i have a sutable answer...its simply because most people dont op a vid if its average, or crappy, therefore most of the recorded opinions are for videos that are of a higher rating

[Ixiel]

The "on a scale of one to ten, seven being average" phenomenom (Spelling incorrect?) is exactly why some people and businesses (including me and mine) use a +2/-2 scale. Nobody feels guilty with a +.1, whereas the same person might feel guilty giving a 6/10.

Ix