Ratings Reloaded - Community Consultation - UPDATED

[smwhitey]

I do kind of like the thumbs up/thumbs down idea, like digg. Maybe we should have that for forum comments too, an easy way to say you agree or disagree with a comment.

I agree. it would give an immediate picture on what people think about a certain subject. and we wouldnt have to do the copy paste thing like this...

oh ya, and bring back feedback.

[ciancama]

for me the present rating sistem is good..but I know that it can cause few problems...
So...For me the solution with a 4 rating point scale is better, but maybe the problems will remain with the people who don't want to understand the importance of the feedback.

[rishaed]

Half the people i play with haven't left feed back. If i have problems deciding what to give them i leave no rating at all because i don't know what they've been doing with their turns. I just take off a star if the person missed 1 or 2 turns nonconsecutively *whew and if they get kicked give them a 1. So if your not sure you don't have to leave a rating. 1 suggestion though take of the team rating for people not on the team because you don't know what they are talking about behind the *team talk.

[ZeroDJoe]

hi lack
feedfack is way cooler then rating
but if do want to stick with ratings, imho:
-5 stars option is good, making it 3 is just like a poor version of the feedback
-atendance rating automatized , new gameplay rating are great ideas
-average rating left calculations great idea as well, and i dont think that people will keep leaving only 5´s cause that takes their liberty to leave good rating when they want too. plus, just a explicite note to make sure everybody understand that 3 is the average rating to be left and a reset of all the rating left so far should do it.

gl

[iLflankU]

-vote for 3 star ratings...but to me nothing ruins a game more (besides cheating) than someone missing many turns and/or deadbeating
-attendance automated
-team rating removed for non team members

what do u do if the person doesnt say a word in chat?
-maybe attitude should be disabled as a rating at that point for ppl who dont talk

AND PPL SHOULDNT TAKE 1 BAD RATING SO SERIOUS. UNETHICAL PLAYERS CAN ONLY SCREW YOU ONCE, BIG DEAL. AVERAGES ALWAYS PREVAIL!

[iLflankU]

I do kind of like the thumbs up/thumbs down idea, like digg. Maybe we should have that for forum comments too, an easy way to say you agree or disagree with a comment.

I agree. it would give an immediate picture on what people think about a certain subject. and we wouldnt have to do the copy paste thing like this...

oh ya, and bring back feedback.

WE ALL AGREE...FEEDBACK IS IMPORTANT! UTILIZE THE WALL! I recommend writing comments on walls and looking at them too. And there should be a limit of 1 comment from player to player to limit spamming or abuse.

[hulmey]

no one uses the wall coz its shit

ive only come acrosee roughly 2 plays with less than a 4 rating!!! This system is proving to be a joke!!

It gives little or no indication as to the player you are playing against, we might as well hae nothing

[Ditocoaf]

I'm looking at other suggestions brought up:

* 4 stars (1-Very Bad, 2-Bad, 3-Good, 4-Very Good) with no "average", so people could typically leave 3 with less controversy.
* 3 stars (1-Bad, 2-Good, 3-Very Good) so that "average" would be more positive sounding as "Good".
* 4 stars with a cap on how often you can rate 1's or 4's. For example, only one "extreme" rating per game to force ratings to be more moderate and therefore (hopefully) accurate.

What do you guys think? I need some more feedback!

Reading this about three stars got me thinking of a different way to describe the ratings.

Were you "unhappy," "satisfied", or "impressed" in your experience with this person? Having the descriptions be from the raters point of view is key to eliminating conflict. You can argue that you were "good" or "bad" in a game, but you can't argue with how satisfied the rater was.

These terms are much more descriptive, while not too specific. I like having less levels, because it will eliminate the "3 or 5 stars" controversy. But if we want more precision (and less accuracy), I'm more terms could be added to make four or five stars.

Unhappy
Satisfied
Impressed

[a.sub]

there is still the problem about unfair ratings in my method this is elimenated

just a random thought, how about in games where there are 4+ ppl we could hav it so that u r required to leave ratings for everyone. BUT for each person, the most common answer is taken. i know that makes no sense so lemme clear it up with an example, lets say i am playing a game with 7 other ppl, 5 leave me 4 star ratings, one leaves me a 5 and the 7th guy is biased and gives me 1 star, well now the most common answer is 4 stars so i get 4 stars. that way biased ppl would basically hav their vote thrown out, and if everyone thinks a guy is a nut then he will get a low rating anyway.
i also think we should go with a 3 star system but instead of three stars we should hav it so u can check mark one of three boxes labeled
[*] poor
[*] average
[*] good
that way the "average" is predetermined and every one is on the same page.
what do u guys think?


also i think we should have "flags"
so next to ur name u can get flags for
[*] deadbeating
[*] breaking a truce
[*] cheating
and anything else u like this way if someone is a habitual deadbeater we will know

[PLAYER57832]

* 4 stars (1-Very Bad, 2-Bad, 3-Good, 4-Very Good) with no "average", so people could typically leave 3 with less controversy.
* 3 stars (1-Bad, 2-Good, 3-Very Good) so that "average" would be more positive sounding as "Good".
* 4 stars with a cap on how often you can rate 1's or 4's. For example, only one "extreme" rating per game to force ratings to be more moderate and therefore (hopefully) accurate.

What do you guys think? I need some more feedback!

My vote is for 3 ranks, but neutral instead of "average" or "good". Just like before.
BUT, since we cannot add comments, a set of "check boxes" for just the nuetral and negs, specifying why. And, they should be limited. (cicero has a thread on this, so I won't repeat)

But a Question? Why does it matter if people leave "extreme" ratings?

That so many people leave all 5's (I more or less include myself now), really means that people are not worried about the distinction between "average" and "excellent". The truth is that over 95% of the CC community make for nice, pleasant games with reasonable chat (including no chat .. chat is nice, but optional). The other 4.999% may miss a few too many turns, may not be quite as polite as some would like, etc, but are folks we would play again (cannot expect perfection all the time ...even from the nicest, friendliest, best players)

We need a way to pick out the few jerks.

The other issue is style and skill. Personally, I think people who worry too much about skill more often than not wind up being jerks. There are exceptions, but if you are going to get upset because someone does not play the way you like... folks will just not play with you after a while (in CC and in life!). There are some issues for the higher ranks (the REALLY high ranks ... nothing below a major), but, ironically, they are usually not the ones complaining about "no skill ratings".

EDIT: Due to popular demand I've added a 5th point to the plan...

Problem: Attendance should be automated, not a rating!
Solution: Add attendance stat to player profile, remove it from ratings. >> discussion topic <<

Love it!

[ImperialPower]

I think use the regular scale, but instead of having 5 be Excellent, it should be, Very Good, or Perfect, or Near-Perfect. So it would look something like this.

1-Consistently exceptionally bad, might ignore comments on behavior
2-few faults, could be improved to be better, but person isn't trying to be disruptive
3-fairly mute, no mistakes, but no plusses either. A complete neutral.
4-has some good times, and is sometimes-usually good at said rating (be it attitude, fair play, etc etc)
5-consistently good to play with. barely-no mistakes or faults. Fun to play with, and definetly a worthy addition to the game.

Keep in mind this is only for Attitude. Fair Play would be scaled differently, as 5 being no mistakes, and as the mistakes get more severe, the rating gets lower and lower. Same with Attendance.

So, to wrap it all up: Attendance+Fair Play start with 100% (5) and count down with the mistakes made. Attitude is determined by the scale said above. What do you think? This is what I've always used, and my scale on rating.

[Mr_Adams]

I don't see why you didn't just make it so you couldn't see the feedback somebody left you until a week after the game is filed away, or whatever it is, and make it so that feedback must be left in that 7 day period. that way you couldn't leave retalitory feedback, because you can't leave some1 feedback once thier feedback is visible.

[Kitchen_Fork]

I THINK ITS FINE AS IT IS

1.....VERY BAD
2.....BAD
3.....AVERAGE
4.....GOOD
5.....VERY GOOD

[vsmith]

just one comment as a player who wants to have a good rating - when players leave me a 3 for fair play or attendance, my question to them is, "I was putting out 100% effort, what could I have done to get a 4 or 5? attendance for example, when i play a game then rate my opponents, I give a 5 to someone if I have no complaints - the player was there for every turn and played b4 his allocated 24 hours were up... what else could he do? fair play - if i don't know of anything the player did that was unfair, how can I justify in my mind giving anything other than a 5? if the system was set up so that 3 was acceptable, 4 and 5 were meant to be for exemplary actions, 1 and 2 were meant to be for below average actions, i could more easily give someone a 3...

[ctgottapee]

the multitiered ratings do not accomplish anything as each player's view of 'very good' is different; it needs to be simplified further.

the problem with current ratings is there hasn't been a clear definition of what exactly the point of them are.
>most object as they see ratings as a personal thing, a pride or tabulation of skill/effort
>others see them as a way to avoid shitty people who happen to play risk well
>and so on

you can't have a singular simple rating accomplish all things

and even if you do decide on what the rating is for, the displayed result needs to be discernable. the current system will just place everyone around 4.5 with no discernable reasons for why that number exists.

i described the thumbs up / down system to give ratings a simple meaning, and a simple use; to decide on whether or not to avoid a game/teammate - you can already tell if they have some playing skill by their score, and you can tell their turn making ability if that rating was automated.

my suggested rating basically has two uses that the current rankings/score can not discern:
1. whether a high ranking player is basically an asshole and no fun to play with
2. whether a newb has a low score because they suck or because they are a newb and graciously learning/trying

what else could a rating discern that would be useful apart from the current ranking/score and an automated score based on turn making ability

the solution is simple, and not difficult to implement:
1. a yes/no simplified rating on whether a player gets along well with others, with a neutral option if they are just average
2. an automated attendance score based on whether you make your turns, very simple, playerX makes XX% of turns he has had, and if you want it even further, you can also tabulate how long on average they take to make their turn
3. splitting a player's rank/score into individual games and team games, so you can judge their ability to play as a teammate
4. a comment system with rebuttal comments allowed (comments will not be moderated based on worth, but based on forum posting guidelines, very easy)
and finally keeping the current ranking/score system

those 5 things give you all you need to know in a simplified manner and allow everyone to keep expressing themselves


for moderation of comments, if you get a comment that you feel doesn't meet forum posting guidelines, you flag it (no bitching/whining) and it is reviewed. the merrit of the comment is not moderated, that can be handled by the rebuttal comment.

[maique]

First off, I liked the old feedback system, but understand the problems with moderation, particularly having been in a moderated dispute myself which kept going back and forth.

I understand (at least what seemed to me to be) the sudden shift from the feedback system to the star system. An automated system giving lack and the mods more room to breathe is something i'd push for if i was as overworked with complaints as i'm sure you were by the feedback system. Still, take as long as you can to make up you mind about what to do next. However much people complain about the stars, there's no point in modification after modification after modification and it won't kill anyone to just wait a little. So, think it through and implement something you're more certain of, don't rush the next solution out of the door.

So, having read all 23 pages of posts...



THE SNAPPIER VERSION:

I think you start getting into trouble when you have people rating other people instead of leaving feedback. If some people don't like being judged, more even dislike being rated, i find.

And don't call it "AVERAGE"!!!, call it "NEUTRAL"! That will resolve a psychological issue I think is driving some people to be unhappy. On this topic, I was interested in the suggestion of having the rating relate to the rater's experience of the game/experience of that user, rather than rating the users themselves.

You shouldn't rate so many things. There are other parts of the site already "rating" the player and you're doubling up on the effort by adding a completely subjective second rating for overlapping catergories. Don't forget that people already have a rank and, now, medals.

Finally, feedback was useful, and if ratings need to stay, they should try to be at least as useful as feedback. emulating the good aspects of feedback as much as they can. 1, 2, 3, 4, 5 stars!, you were really just begging for trouble, weren't you? heheh. Ratings are too complicated where they needn't be (too many catergories and nuances left to interpretation, even if you try to further define them) and too simple where they shouldn't (it's too easy to leave a rating, particularly a negative rating, when you don't need to think about it, let alone justify it, thereby promoting abuse). So K.I.S.S., but don't make the jerks' life that easy!



THE LONG, DRAWN-OUT VERSION:

Ratings should be positive or negative. No neutral. And do away with the stupid ratings medal, what the heck is that all about?!?! Positive feedback is self explanatory and needs no written feedback/sucking up. Negative feedback needs to be justified. I'd personally have negative justification be written as opposed to tagged as tags make it to easy to just click "whatever" just for the sake of being negative, as opposed to actually having to justify yourself (emulating the issue of over-simplified negative feedback already found in the star system). Written feedback would be restricted to a lot fewer characters than previously allowed in the Feedback System (a one liner would suffice!), thereby reducing the colourfulness of abusively insulting comments (it's harder to insult one, one's family, one's country, one's society, one's culture and anything else that's not game-related and that pops into the mind of the ignorant if there is simply less character-space to fill, or do cowards only do that through PM?). As it happens now, ratings/reviews/feedback would only show up a few days after the game's end, doing away with retaliatory feedback (you'll never elimite bushian pre-emptive strikes, but no system is perfect). The result would be a nice little couple of numbers next to the player's screen-name like you had with feedback.

If you need to have a five star system, don't break it up into different catergories. in fact don't even call it anything other than a user's "Rating". And don't try to define the gradation from 1 to 5, simply define the extreme stars by saying something like 5 is for "this player is increadibly amazing" and 1 is for "this player is not just bad, he actually turned this game into something i'd rather not have experienced by being horribly offensive or wasting everyone's time by consistently benifitting the same other player or by constantly missing two turns and then coming back without giving any reason", or simply 1 is for "this player is waaaaaaaay beyond bad". In case you missed it, that would make 3 pretty "NEUTRAL", instead of the possibly-derogatory "average".

Things related to people missing turns and such should be on the profile page, if anywhere. I'll leave the implementation to your thinking. Beware of mixing this with the overall rating of a person. People live in different timezones and some of us do have a life beyond ConquerClub from time to time. If people want to judge others' speed of play (i.e. how available they are to play and how much of the rest of their life they are willing to give up to play risk on-line and how long they take between turns) they should play speed-games and freestyle-games. Even then, how many times a person plays in a 24 hour period shouldn't have a catergory of its own and should, rather, be diluted into the rest of the feedback (whatever shape or form it finally takes), in my opinion.





Well, that's what I thought, if anyone could be bothered to read all o'that.
Apart from that, ConquerClub is a great site, thanks for the idea, the implementation and the effort. I hope whatever rating system we end up with is as usefull as the Feedback system, but I'm sure I'll live with whatever comes since what I really like is the gameplay. With regards to the ass-holes who do their best to ruin perfectly pleasent matches... just have to learn to ignore. Silence is better than being drawn into a futile discussion with an idiot. Plus, there's always the foes list.


P.S.:I was somewhat interested by a comment about a Foes lists entry-count - an anonymous count of the foes in one's list and the lists one's a foe on. Once more, if anywhere, i think I'd have this on the Profile.

P.P.S.: Sorry to the people whose ideas I quote without attributing authorship, after 23 pages of posts I sort of lost track and then I couldn't be bothered to go back and sift through it all.

[PLAYER57832]

LOVE that attendance is not off the subjective ratings.
HOWEVER,
It seems that what I think most people saw as a "brainstorming excercise" suddenly became set tags wihtout any further community consultation.

Last I checked Cicero's thread, the few people who posted pretty much thought things should be simplified.

I appreciate the effort, but this is now a HUGE mess. There is no guidance on what everything means. Critical elements were not included and a lot of unneeded information was included. And, a lot of the later ideas in this thread suggested other ways of dealing with this.

Because things are not defined and explained, (even where explanations are possible), it will lead to even more complaining and misunderstadings.

Also, you left in the "fair Play" ... which pretty much gave everyone grief, (I have yet to see anyone who really thought this one was clear) and added an even more ambiguous rating "Game Play". PLUS, the game play attributes include all sorts of things that ought not be there .. such as team play, attitude, etc.

IN short, I used to rate EVERYBODY, I have a gold medal for rating, but I don't see myself rating any now! Much easier than dealing with the grief from "mis labeling" someone!

[ctgottapee]

as i mentioned before, the approach is backwords.....

it SHOULD NOT be: let's have ratings, and then try to define what they are

it SHOULD BE: let's define what ratings we want to see and what we want certain ratings to mean, then develop a ratings system to reach that goal
(example> 'i want to know whether a player deadbeats often' solution: 'automated attendance score that shows a percentage of turns a player missed according to his overall turns'

what do you want 4.3 to mean?? right now it means absolutely nothing
i'm currently a 4.8, what does that mean about me? am i identical to all other 4.8 rated players.... hardly....


AND the comments have to come back; i just don't get was it so hard about them
reduce the comment size to prevent the diatribes, allow rebuttal comments, and moderate them the same way you moderate forum posts, based on the content and not the relevance.

[maique]

by ctgottapee on Sun Jul 27, 2008 1:09 am

[...]
AND the comments have to come back; [...]
reduce the comment size to prevent the diatribes, allow rebuttal comments, and moderate them the same way you moderate forum posts, based on the content and not the relevance.

simple and to the point. i second the above.

and, by the way, the rest of his comment makes perfect sense too.

[Ruben Cassar]

We should be able to delete the response comment because I made a mistake and wrote a response for a rating when I intended leaving it for another one and now I cannot delete it.

[Scott-Land]

We should be able to delete the response comment because I made a mistake and wrote a response for a rating when I intended leaving it for another one and now I cannot delete it.

You should only be able to respond to the rating that are left for you Ruben- in which you can click [respond] to change. Or in your case- backspace to delete the response then click [respond]. It should clear what you wrote...

[Ruben Cassar]

We should be able to delete the response comment because I made a mistake and wrote a response for a rating when I intended leaving it for another one and now I cannot delete it.

You should only be able to respond to the rating that are left for you Ruben- in which you can click [respond] to change. Or in your case- backspace to delete the response then click [respond]. It should clear what you wrote...

Yes, you're right. My bad.

[Quicksigns]

Once again, we have gone from bad to worse. Just go back to the original. Like I said before, why did you try to change something that wasn't broke in the first place?

[walnutwatson]

I would suggest getting rid of the ratings system all together.
If a player really pisses you off to the point where you never ever want to play with them ever again in your entire life, then isn't it reasonable to assume that you'll remember to put them on your foes list or whatever.

And seriously, we're playing a board game here, how do you people get so upset over it? Personally I quite like having arguments/abuse thrown at me during a game, after all it is ONLY a game and all that sort of stuff just makes you want to win even more, for gloating purposes.

[Ditocoaf]

How was your experience with this player (in category X)?

Unhappy
Satisfied
Impressed

Rating your experience, rather than the other player per se. Solves a lot of issues, quickly.