WCM Network: 
 


Go Back   mmorpgforum.net > Machinima > WarcraftMovies.com > Suggestions
Register Projects FAQ Members List Calendar Search Today's Posts Mark Forums Read

Reply
 
Thread Tools Display Modes
Old 02-21-2007, 01:47 PM   #1
MaceMan
Initiate
 
Join Date: Apr 2005
Location: Texas, USA
Posts: 10
MaceMan is unknown at this point
Rating system suggestions

I remember back when the rating was a 1 to 10 system. It was known that it needed fixing, and it was fixed. I'm glad that we are on a better system now that uses 1 to 5, and has multiple elements to grade. It is now possible to more accurately rate a movie. The 1 to 10 system was a problem, and now it's fixed, but now I know that it was only half the problem. The people contributing the ratings are the other half. People grade all 5 if they were entertained, and all 1 or 2 if they weren't. I'll get back to this in a minute.

Another problem is that it's difficult to tell what each point on the scale actually means. Does 3 mean you didn't like the movie or does 3 mean it was average? I personally believe that 1 means it was bad, 3 means it was average, and 5 means it was one of the best movies ever. It seems that others think differently. I personally feel kind of bad when I give my honest ratings, because for me, 3.5 is an above average movie. I feel bad because everyone else is voting 5 and it looks like I'm trolling or didn't like the movie. There is a solution to this particular problem:

Problem: Each point on the 1 to 5 scale has a different meaning to different raters.
Example: I go to rate a movie, and I thought it was a o.k. movie, but nothing special. Do I rate it a 3 for average or do I rate it higher? This other guy thought it was average in his comment but gave it a 4, what do I rate?
Solution: Put labels on the scale, such as "1 - Poor", "3 - Average" and "5 - Outstanding". I'm not trying to suggest what the labels should be, I'm just suggesting that there be labels. Labels would make it so that me and the other people rating the movies are on the same scale.


This brings us to another problem. Even with the new labels the average voter will ignore the labels. When the 1 to 5 system came out, I thought we were finally going to get accurate ratings on the movies, but I was wrong because I didn't count on the average person. I believe the average person is not a reliable source of precise ratings. Many people ignore the different criterion and just rate 2 or 5 in all four categories without thinking about the different categories.

Problem: People rate very high if they liked the movie, and very low if they didn't, ignoring the multiple criterion.
Example: Average Joe liked the new PvP movie that just came out. He gives it all 5s.
Example 2: Average Joe didn't like the new storyline movie that just came out. He thought it was a waste of his time, so he gives it all 2s without weighing the different criterion.
Solution: Make a two point rating system. "I liked the movie" and "I didn't like the movie". This isn't ideal, but it's accurate with those that rate on impulse and with those that don't consider the meaning of a 2, 3, or 5.


Maybe this isn't accurate enough, or perhaps this leaves out the many voters that do think about their ratings before they click 'submit'. What about trusted raters? In the mainstream movies, professional movie critic's opinions have a place, as do average people's opinions. Why not translate that into machinima? This already partially exists with the fame system, but that's for weeding out trolls. What if people that prove themselves to give good ratings (good as in not always 1 or 5, maybe explains their ratings, maybe puts constructive criticism, basically not a "noob") get awarded with the ability to have their ratings be "trusted" ratings? The "trusted" ratings would be along side the normal ones. Maybe the normal ratings would be 1 to 5, only that would be the result of averaging "Liked it" and "Disliked it", and the 'trusted' ratings would be 1 to 5 as a result of the average of all the other trusted raters averages of the different criterion.

Problem: Many people rate on impulse and emotion, but some rate maturely.
Example: Average Joe rates a movie he liked as a 5-5-5-5, Mature Joe rates a movie he liked as 4-4-3-5. The professional-like rating is mixed with the impulse rating.
Solution: Average people rate with my previous suggestion (liked it/didn't like it), and approved 'critics' rate with a more precise scale, like the one currently in place for the general public.



Thank you for reading my ideas. I encourage you to come up with modifications of my suggestions.
__________________
__

Last edited by MaceMan : 02-21-2007 at 01:50 PM.
MaceMan is offline   Reply With Quote
Old 02-21-2007, 02:26 PM   #2
godliest
Senior Member
 
godliest's Avatar
 
Join Date: Apr 2005
Location: Sweden
Posts: 396
godliest is notable
Oh well, oh well (why did I start like that?), these are alot of good ideas and I agree with u about how people rate. I myself rate a movie 5/5/5/5 if I like it. U took a PvP movie for an example here, but the point with PvP movies is that u can't really decide if a PvP movie was creative. I mean since when did it become creative to kill other players and film it, it has been done since the start of WoW.

WCM is not all too full of average Joes but there are a few. I would say that the most peoples rate the movie like the person before did as I they don't want to be remembered as the guy who didn't agree with the rest. So if a movie has a average on 4.99, a person who thought that it shuld get like say 2-1-4-3 doesn't really dare to do this because of fear that he will get flamed.

There are also alot of flamers on WCM who just give u the comment "..." and 1-1-1-1, these are truly annoying for the creator of the movie. These guys are just annoying.

Some nice ideas how to "stop" the "average Joe" from rating like he feels and make it more like he's rating the way it was intended.
__________________
Give me + to my posts
or be destroyed

Albert Einstein Quotes
"Gravitation is not responsible for people falling in love."
"Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius - and a lot of courage - to move in the opposite direction."
"Two things are infinite: the universe and human stupidity; and I'm not sure about the the universe."
godliest is offline   Reply With Quote
Old 02-23-2007, 09:59 AM   #3
MaceMan
Initiate
 
Join Date: Apr 2005
Location: Texas, USA
Posts: 10
MaceMan is unknown at this point
Quote:
Originally Posted by godliest View Post
I would say that the most peoples rate the movie like the person before did as I they don't want to be remembered as the guy who didn't agree with the rest. So if a movie has a average on 4.99, a person who thought that it shuld get like say 2-1-4-3 doesn't really dare to do this because of fear that he will get flamed.
I agree.

Quote:
Originally Posted by godliest
Some nice ideas how to "stop" the "average Joe" from rating like he feels and make it more like he's rating the way it was intended.
I wouldn't say I'm trying to stop people from saying what they think, I'd say I'm giving suggestions on how to reduce bogus and inappropriate ratings that negatively impact the rating system. There may already be some of this being taken care of with the rating 'weight' system, but how effective this is at removing bogus ratings (if someone rates a lot, do they get a lot of weight? posting a lot of ratings doesn't make you an honest rater) is unknown to me.


I'd like to add to my original post that I think PvP movies can be difficult to rate. Creativity on PvP movies is one example, although they can be creative depending on the type of PvP movie they are, see the movie Unknown III for an example of a creative PvP movie. I usually rate a "3" for "average creativity" on creativity if it's just fighting, since the average PvP movie has little more than fighting. I think the idea (which is on the front page poll) of replacing "Content" with "Skill" is a good idea, but remember some people think that putting only your best fights on your movie means you have no skill. According to this, if you put in your best fights, you suck, but if your movie has your past nine PvP fights, it's going to be boring.
__________________
__
MaceMan is offline   Reply With Quote
Old 02-23-2007, 10:20 AM   #4
unska
Senior Member
 
unska's Avatar
 
Join Date: Jun 2005
Location: Finland
Posts: 189
unska is unknown at this point
I got a nice idea of this topic, thank you!
unska is offline   Reply With Quote
Old 02-23-2007, 01:59 PM   #5
godliest
Senior Member
 
godliest's Avatar
 
Join Date: Apr 2005
Location: Sweden
Posts: 396
godliest is notable
I added stop between two " because I couldn't come up with a better word so I used the word stop. It shouldn't be seen as stopping som1 from doing something, it's more like making them rate as intended. If you understand....
__________________
Give me + to my posts
or be destroyed

Albert Einstein Quotes
"Gravitation is not responsible for people falling in love."
"Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius - and a lot of courage - to move in the opposite direction."
"Two things are infinite: the universe and human stupidity; and I'm not sure about the the universe."
godliest is offline   Reply With Quote
Old 02-24-2007, 06:32 PM   #6
Z.x
Member
 
Join Date: Jul 2006
Posts: 31
Z.x is unknown at this point
I've given some ideas before but I'm more of a "don't mess around and just Fix it." kind of person, so my best suggestion was to let the authors remove unjustified ratings/trolling from their movies' comments, in some form. There's many ideas on how the accuracy of ratings can be improved, but it's very difficult to make a solid system without errors. It won't really stop some of these ratings/comments.

Quote:
Originally Posted by MaceMan View Post
I'd like to add to my original post that I think PvP movies can be difficult to rate. Creativity on PvP movies is one example, although they can be creative depending on the type of PvP movie they are, see the movie Unknown III for an example of a creative PvP movie. I usually rate a "3" for "average creativity" on creativity if it's just fighting, since the average PvP movie has little more than fighting. I think the idea (which is on the front page poll) of replacing "Content" with "Skill" is a good idea, but remember some people think that putting only your best fights on your movie means you have no skill. According to this, if you put in your best fights, you suck, but if your movie has your past nine PvP fights, it's going to be boring.
The vast majority of raters have no idea how difficult it is to fraps good fights vs well geared players without interruption, and heavily edit a video with good editing/effects that doesn't turn out "overedited". So I see way too many inexperienced people give the inaccurate ratings you described in the original post for this reason. I think that people who have made movies before should be given a much greater weight to ratings, or some form of influence like that, while ones who haven't, shouldn't be allowed to rate certain categories?... Something like that. I know the 'fame' system works somewhat like this but don't think it gives enough influence to movie makers, over the 'average joe' rater.

That said, I know WCM is trying to improve on this category and appreciate the effort; it's better than it used to be. Keep working on it.
Z.x is offline   Reply With Quote
Old 02-27-2007, 09:15 AM   #7
Hentai
Senior Member
 
Hentai's Avatar
 
Join Date: Nov 2005
Posts: 102
Hentai is notable
people keep discussing rating sistem xP for what? Nothing changed and its not a good rating sistem after all...
Hentai is offline   Reply With Quote
Old 02-27-2007, 10:45 AM   #8
godliest
Senior Member
 
godliest's Avatar
 
Join Date: Apr 2005
Location: Sweden
Posts: 396
godliest is notable
Why isn't it a good system? The thought behind it is good, and if something change to the better thanks to this discussion then it will be good...
__________________
Give me + to my posts
or be destroyed

Albert Einstein Quotes
"Gravitation is not responsible for people falling in love."
"Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius - and a lot of courage - to move in the opposite direction."
"Two things are infinite: the universe and human stupidity; and I'm not sure about the the universe."
godliest is offline   Reply With Quote
Old 02-27-2007, 11:25 AM   #9
Gedan
Moderator
 
Gedan's Avatar
 
Join Date: Sep 2006
Posts: 631
Gedan is unknown at this point
Quote:
Originally Posted by Hentai View Post
people keep discussing rating sistem xP for what? Nothing changed and its not a good rating sistem after all...
I don't know what you're applying, we do tweak the rating system 2-3 times a year based on what the community wants and with the most resent Fame-system we have been able to lower the Comment/Rating hijacking with ~50%. I do think we have an accurate "top rated this week", if you look at the movies up there I think everyone can agree they deserve to be there?.

To comment on MaceMan outstanding post about this issue

We do know that many of our users just like to click 1 (0 if they could) if they dislike it and max out if they saw something they like, one solution you mentioned about using a two point rating system might help and we will look in to it. We will absolutley ad the lables you talked about, great idea.

Our current system is based on weight on activity - People with low rating activity will not effect the average rating as much a person with high activity would. This system comes with anomalies but we do belive after some time members mature and woun't be satisfied rating 1 or 5 so their votes will be more accurate and also with more weight they will help create a fair average for the movies.
__________________
Mod it
Gedan is offline   Reply With Quote
Old 02-27-2007, 03:50 PM   #10
Hentai
Senior Member
 
Hentai's Avatar
 
Join Date: Nov 2005
Posts: 102
Hentai is notable
Quote:
Originally Posted by Gedan View Post
I don't know what you're applying, we do tweak the rating system 2-3 times a year based on what the community wants and with the most resent Fame-system we have been able to lower the Comment/Rating hijacking with ~50%. I do think we have an accurate "top rated this week", if you look at the movies up there I think everyone can agree they deserve to be there?.
there is stil a big problem, do I really need Audio quality or even instead of content something like "Player skills", someone had already posted in other threat what he thought of that, and I agree. Some kind of movies should have another tipe of voting sistem : ) People keep voting high creative in PvP Movies that are good, but no creative at all...

I gave and ideia in other post about people with more fame, above 40 or 50 could have more % when they vote 5
Hentai is offline   Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


Forum time (GMT) 01:25 PM


Powered by vBulletin® Version 3.6.12
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.