Star Ratings in the Announcement Forum

[Phantasmagoriat]

Star Ratings in the Announcement Forum

Issue: Whenever possible, I think publicly leaving star ratings in the Announcement Forum should be discouraged, in favor of descriptive, meaningful comments...

I wouldn't be surprised if this has been discussed before, but I still hesitate to bring it up because a lot of people do it (and I used to do it). It almost seems natural to leave a rating, and surely you want to communicate your opinion to the creator. But I'd like to say that star ratings are not good for feedback, even if you leave a comment as well (you effectively nullify your comment with a rating... more on this later...). In fact, ratings are generally intended for statistical purposes, so they have no place in feedback. I realize this can't really be enforced, so I'm not proposing this be a rule, but hopefully someone reads this and it catches on. Also, this is just my opinion... I'm not targeting anyone... and I'm not forcing anyone to do anything...

A Beacon to Everyone
The forum is for discussion of videos, and the more discussion the better. However, when leaving a public star rating, it sends a signal to everyone in the forum; a kind of 'beacon' to tell everyone else what to expect... and to a certain extent... even what to think (admit it, people are more likely to agree with someone else' opinion than they are at forming their own. If not, they at least make comparisons). So when other people see the rating, they may already make up their mind on downloading the video. This drastically cuts down on the amount of discussion, which is obviously counter-productive. Just think about what happens when someone announces a video and the first responder displays a low star rating. Even just a 3/5 signals to everyone: "Mediocrity Here... don't download" which can really kill a thread. (in some respects, a 3/5 can be more of a thread-killer than a 2/5) You may think you are doing everyone else a favor by telling us which videos are worth downloading and which ones aren't, but that is not your decision to make because everyone's opinion is different.

My Rating System is Different Than Yours
I don't care if you have the most elaborate rating system ever used by mankind. We don't all use the same system. A 3/5 may mean a lot coming from you, but other people may take it as an insult. Similar differences regarding 5/5's. However, when people see your score in the announcement thread, they put your 3/5 on the same level as their 3/5 which could be the difference between downloading a video they enjoy, and not downloading a video at all. That would be a shame.

Your Mood/Biases
As stated earlier, we all have different opinions, and if you have a particular distaste towards a certain music/anime/etc... that is going to be reflected in your star rating (even if you try to account for that, I doubt your results will be fair... I know if I watch an AMV with music I like, it tends to have a slight *edge* over other videos... and it's impossible to reintroduce the same *edge* to videos with music I don't like...). Or you may have a particular set of expectations from another editor, which completely influences your mood. I know I've let that influence my score. Your public star ratings don't account for these biases, and neither can people on the boards. The problem is Ratings are not very descriptive. However, Comments are great for airing these biases since Comments are specific-- they call things as they are-- unlike a star rating which dwindles down complex thoughts into a single numeric value. (And who knows, when explaining your bias, you may enlighten other members to different genre's of anime/music/editing styles etc...)

Also, we are all full of biases that we never think about, so a public star rating is not very fair to the creator. I would rather see someone explain why they don't like the music than see a star rating that's been influenced by this bias (the worst part is you would never know if a rating is severely bias). A better indication of a rating would be to see the overall star score from the members page, which is a bit more accurate since it averages out biases to a certain extent... though that has it's problems too. The creator can check that themselves. If you really feel like communicating your score to the creator, use a quick comment, but try to justify it with a description. If you can't do this, I would say... just omit the rating. A comment by itself is much more meaningful.

--------------------------------

A Study on the Effects of Grades/Ratings:

Spoiler :

(I don't have the article anymore, but it was by Dr. Ruth Butler,1988)
Reading this study completely changed my views on ratings. In fact, if you look at my profile, I have never given a formal OP ever because of the numeric scores involved, but I have given plenty of lengthy opinions-- usually in the form of quick comments, posts in announcement threads, or occasionally PM's (mainly beta's). Then again, I'm a bit more extreme in terms of my views on ratings, as you'll hopefully see why below.

The study monitored the performance of 3 groups of students over the course of several exams/quizzes/tests. Whenever Group_A wrote a test, they got feedback in the form of a Percentage. Whenever Group_B wrote a test, they did not get any type of rating, just Comments (what they did good... what they did bad... and what they could improve upon). Group_C received both a Percentage and Comments.

The study found *outstandingly* that Group_B had the greatest improvement. And I'm not just talking about averages... I'm talking *globally*. Which means not only did the low-performing students improve, but the students that were already high-performers improved as well. In Group_C, only some of the lower-performing students improved (but not all); the high-performing students stayed the same. Dismally, Group_A had zero improvement.

[wordy technical explanations]
There were a number of explanations, but it's pretty clear that grades/ratings have little to no impact on improving performance. Group_B improved the most because nobody knew exactly what level they were at-- they just knew what they did good and bad, but there was *always* room for improvement, even if performance was high to begin with (The natural response is to use the suggested improvements). Some low-performers in Group_C improved because they were actually given feedback on how to improve (unlike in Group_A, that's a no-brainer); Other low-performers in Group_C did not improve because they were content seeing their low grade, and saw no need to improve (kind of sad). The high-performers in Group_C did not improve because they could see their performance was already high, and felt no need to improve further. Group_A did not improve at all because they were given no feedback whatsoever regarding how to improve.

Basically, Ratings do nothing but reinforce a student's current level of performance. Comments tell students how to improve, but only if the student is inclined to use those comments. I can see these messages being extended to any situation involving some sort of assessment-- like commenting on AMVs.

To sum up, if you want editors to improve:

• Avoid including Ratings in your feedback-- they are not helpful to anyone
• Commentary is the only way editors will know how to improve. (aside from solitary learning...)
• Avoid giving both Ratings and Comments at the same time-- otherwise the editor will be fixed on the Rating, and overlook valuable Comments.
• A comment by itself is much more meaningful.

--------------------------------

So, what do you think? Is it okay to leave star ratings in announcement threads?

[Arigatomina]

People used to do the Up/Down thumb thing before we had the stare scores. I guess a 3/5 would be like the old 'meh' one-liner posts since it's in the middle. They've always done this when they had nothing to say about the video itself. To me, it's not really about feedback or even the video itself. It's about the editor, the person posting the score, and those who view the thread. People post "I approve" one-liners to their friends just to let them know that, yes, they did watch the video. They post low scores and high scores (with no comments) because they think name-recognition will make their verdict affect others and often they're right. It's less about telling the editor what you thought, and more about letting other people who view that thread know whether it (and the editor) has your seal of approval. So for them, posting a star score does exactly what it's supposed to do - influence other people who might be thinking about watching that vid, and telling the editor as quickly as possible where s/he stands in their eyes.

I don't think most announcement threads are about improvement. They're just about getting attention, getting the video viewed and getting others to read their name if not remember it. For them, the score tells them exactly where they stand. Any positive comments are just a bonus. Negative comments are the ones that lead to discussion, but most editors don't want their flaws pointed out publically where they have to defend themselves and argue about it.

As far as scores and comments affecting the person who receives them, maybe it's a generational thing. I've gotten a lot of ops and I never focused on the score unless it was all 1s flamefest thing. I do like seeing my ops box in the yellow on my profile, but with individual ops I don't even read the scores unless the reviewer includes them in the comment boxes. I always include them since I explain how I arrived at each score. I wouldn't be surprised if people I've left reviews for focused too much on the scores. I know the op exchange is all about scores and number of ops, with hardly anyone actually interested in feedback. But that won't stop me from writing ops. Those scores serve a purpose, they're there for the top 10 list and for that little box on the editor's profile page. That little box in particular is underrated. It tells the editor (and potential viewers) what that person's weaknesses are over their entire body of work and not just one particular vid. Unless you PM a creator and give them a comprehensive review of their entire body of work, you're not providing the information that little box gives. And since few people are going to do a comprehensive review, especially for the more prolific editors, the scores will continue to serve a purpose.

I agree that comments are better than any score could ever be, though. The problem is some people don't want advice or even praise, they just want another op to add to their count, another post to make their annoucement thread bigger, another 10 to bump their vid up on the list, etc. When people don't care about comments there's no reason to bother typing one up.

[JudgeHolden]

Well, your logic would dictate emlimiting star ratings all together (which I would be for), because even if you click on the link you still see a star rating on the video's page ... Ok, only if you are a donater, but still it has the same effect.

[Phantasmagoriat]

I agree that comments are better than any score could ever be, though. The problem is some people don't want advice or even praise, they just want another op to add to their count, another post to make their annoucement thread bigger, another 10 to bump their vid up on the list, etc. When people don't care about comments there's no reason to bother typing one up.

That's precisely what I think is wrong with everything. It's hard to change the mentality of everyone as a whole, but it's easy to change what you as an individual do. Focusing on comments & discussion instead of ratings & attention is at least one shift people can make. I think it would be a good start. Think what it would be like if every response in the announcement forum encouraged discussion instead of blunt evaluation of videos.

Well, your logic would dictate emlimiting star ratings all together (which I would be for), because even if you click on the link you still see a star rating on the video's page ... Ok, only if you are a donater, but still it has the same effect.

And that's where the dilemma is, because I still like having a system that makes it easier to find good videos, and star ratings serve that purpose. I just don't think the announcement forum should be the place for it, though admittedly, it is one of the primary places I find videos.

[Nya-chan Production]

You can't change the people - unless you'd ban them, but that's not the way to go, obviously

If forbidden they would use something else - and come to mind...

[Panky]

I think people usually use these because it's easier to convey your valoration of the video through numbers, more like in cases when you have practically nothing to add but "I really liked it and can't see anything wrong with it". Something to add more sustance into it.

[Phantasmagoriat]

You can't change the people - unless you'd ban them, but that's not the way to go, obviously

If forbidden they would use something else - and come to mind...

I know what you mean, although I'm not saying we forbid public star ratings, just encourage people to use more descriptive means to say what they think. Actually I don't mind the thumb icons. Truth be told, the best way to get people to improve is through encouragement (contrary to the bashing method some people use), and star ratings do nothing of the sort, particularly when a lot of 3/5's get thrown around. Yet the icon does provide encouragement (people generally don't use the ). As sappy as it sounds, Positive Feedback is still the best way for people to recognize what they have done right, so they continue to do those things right in the future. It's called Reinforcement. Generally accepted as fact since the dawn of modern psychology...

So yeah, I agree you can't change everyone, but even a small number can make a difference.

I think people usually use these because it's easier to convey your valoration of the video through numbers, more like in cases when you have practically nothing to add but "I really liked it and can't see anything wrong with it". Something to add more sustance into it.

x2 <---point in case
I don't like stuff like that. Although it's a tricky situation because honestly I would rather see a bunch of x2 x3 x4's than no responses at all... and people are more likely to respond that way out of convenience...

[dreamawake]

So basically, let's limit our already limited amount of freedom of speech even more!

Fuck that. I'm completely against it.

[Phantasmagoriat]

So basically, let's limit our already limited amount of freedom of speech even more!

Fuck that. I'm completely against it.

So, you like robotic conversion of people's thoughts into numeric star ratings for the sole purpose of saying "I watched this video, and I'm posting in this thread."

If anything, star ratings limit your freedom of speech even more... because they don't describe what you really think.

[dreamawake]

So basically, let's limit our already limited amount of freedom of speech even more!

Fuck that. I'm completely against it.

So, you like robotic conversion of people's thoughts into numeric star ratings for the sole purpose of saying "I watched this video, and I'm posting in this thread."

If anything, star ratings limit your freedom of speech even more... because they don't describe what you really think.

And restricting the use of a rating isn't going to help that problem.

Instead of a rating it'll simply become Shitsux or Goodstuff, even tho that is what most of my ratings are anyway...