PDA

View Full Version : Guideline for rating movies.


cellian
05-26-2007, 02:37 AM
Hi,
There's one thing about the ratingsystem that concerns me since it has been updated. People are unable to rate "fair". What I'm trying to say, is that people are rating movies they like with 5 stars and those they hate with 1 star, even if it is impossible for the movie to get that rating.

For instance Drifting 3 http://www.warcraftmovies.com/movieview.php?id=40621

Let's say I loved the movie, so I instantly give it a rating of 5, but that's simply wrong. Because if I rate fair, it wouldn't be possible.
Let's take a closer look at how I would rate:

Content: I really enjoyed every fight of the movie = 5 stars.

Creativity: I liked the movie, it's an ordinary PvP movie, it has an intro, an outro, but still nothing special = 3 stars.

Editing: There's a lot of editing in it which fits to the music. Since I'm a fan of a bunch of effects = 5 stars.

A/V Quality: For me, movies with a resolution of 640*512 are absolute average. You can clearly see everything if the UI is clean. Everything above that resolution would mean 4 stars for me. The only movies I would give 5 stars, are those with a resolution of at least 1024*768 or the similar wide screen version (which I don't exactly know). This one is above the average = 4 stars.

(5 + 3 + 5 + 4) / 4 = 4,25

Now if we look at the comments, there are a lot of people rating it 5 or even worse, someone rated it 1 and that's not OK. Even if you love the movie the quality wouldn't deserve 5 stars, neither the creativity. Giving it 1 star is much more unlikely, because the quality is still good, even if you hate everything else.

What I would like to see is some kind of rating guideline which appears when somebody wants to rate. Or something like a notification if someone is about to rate a movie with 1 or 5 stars.
Thx,

oh and sorry for my shitty english...

godliest
05-26-2007, 11:17 AM
Yeah... Also recognized this, people doesn't seem to truly understand how to rate a movie. You got 4 different things to rate on a grade from 1-5, simple? No, doesn't look so. The best example ever is: "The editing sucked but the PvP was awesome" rated 5.0.

Olibith
05-26-2007, 08:18 PM
Or maybe an obligatory comment for each criterion, that could make people think a little more before rating.

Or an IQ test to balance rating weight. :D


Anyway this topic deserve a straight 5/5/5/5.

godliest
05-27-2007, 09:04 AM
Yeah I agree with Olibith. The editing, text, content, quality and creativity sucked, but the movie was awesome! 5/5/5/5.

However, to be serious. I think that authors should be able to see how the person has rated. Since WCM only shows the average score people on wcm also only thinks about the average score while they really should think about each of the criterion. I think the average score should be skipped totally and only showing each of the criterion. Then people would rate it more serious and thinking more.

Also I think ability to write to more text should be given. As it's now it feels like the site is made to write a short comment like "I like" or something while instead I think you should be able to make it a much deeper review if you would like to. As it's now the focus of the site is on the description of the movie and facts about the movie, while I think more focus should be laid on the comments. Making it more forum-like.

I also liked Olibith's idea about a obligatory comment for each criterion. And harder moderating on comments would be nice. Show no mercy for the flaming spammers.

Btcc22
05-27-2007, 11:05 PM
I'm guilty of *sometimes* rating with the average score in mind rather than each of the points.

The reason behind that is because I know only my average score is going to be shown, not each rating point and I feel that sometimes forcing an average score is fairer than rating each point properly. For example, I might feel one aspect of the video outweighed anything in it that wasn't so good, so I might use the average score to give more weight to perhaps the editing score, even if the content wasn't so good.

Anyway, hope that made some sense, and the suggestions made in this thread seem pretty good. I think a rating system overhaul has been due for quite some time.

Olibith
05-28-2007, 09:25 AM
I'm guilty of *sometimes* rating with the average score in mind rather than each of the points.

The reason behind that is because I know only my average score is going to be shown, not each rating point and I feel that sometimes forcing an average score is fairer than rating each point properly.

That makes sense. I also think about the average final rating.
Imagine for exemple a video, funny as hell, made voluntarily in a "South Park" quality way.
Would it be fair to rate it 2.5, even if it's the most entertaining thing you've ever seen ?
Different ratings criterions for each category could be an idea too.

godliest
05-28-2007, 05:05 PM
That makes sense. I also think about the average final rating.
Imagine for exemple a video, funny as hell, made voluntarily in a "South Park" quality way.
Would it be fair to rate it 2.5, even if it's the most entertaining thing you've ever seen ?
Different ratings criterions for each category could be an idea too.

No it wouldn't be fair, but it still would be logical because of the criterion. That's the problem with the average rating. As an example: I loved a PvP movie, it got some sick editing, high quality and original music. However the PvP was terrible, even I could perform better. 1/5/5/5. While I would like to give the movie and a/v on 4,9.

I'd say the best way is that you rate after some different criterion and then check how much a/v score you want. On a scale from 1-10 for example. The movies are then parted up in "Rated for high editing", "Rated for high xxxxx". You get my point?

I know it's like going back to the old system, but you will also add the ability to see more exactly what every person thought about the different criterion.

Gedan
05-30-2007, 08:39 AM
There are some interesting views here on how to further improve the raiting system, we will look in to it. I like the extreme warning when someone goes all out 5555 or 1111.

godliest
05-30-2007, 01:01 PM
I myself would prefer not 1-5, but rather 1-10. Giving a movie only 10s would then be harder and really showing the movie was terrific.

cellian
05-31-2007, 10:26 AM
I can't agree with this "pushing the rating to make it fairer" thing. For example what you said, godliest.

You would rate a movie 1/5/5/5, but you would like to somehow give it a higher a/v rating, due everything else was excellent. 1/5/5/5 would mean an a/v rating of 4. If everybody on wcm.com would know how to rate properly, 4 would be a really good rating.

The problem is, that those who give a/v ratings of 5 and 1 are destroyed the whole a/v rating and also the a/v rating of each criterion. I don't feel bad to give a 1/5/5/5 rating or something similar, because that's what it is made for (as long as people know how to use it..), and if everyone rates the right way the author will see which of his criterions has the lowest a/v rating. By that he knows what he has to improve the next time he makes a movie.

Dwârv
06-04-2007, 03:51 AM
I myself would prefer not 1-5, but rather 1-10. Giving a movie only 10s would then be harder and really showing the movie was terrific.
sounds good, but for shure will not work. people would just rate straight 10 or still 1...
I like the idea of a warning popping up in case you rate all 5's or 1's.
A little step in the right direction might as well be a poll asking : "Do you use your brain - or whats left of it - when you rate?"...something like that.Might help some people start thinking about it.
A basic problem is, I think, that many people simply have no idea what the f*** editing IS. For people like them a fifth categorie in the rating system might be helpful . "I liked the movie" 1-5.

this thread should be recommended on the frontpage, seriously.