OPEN RATING IN PUBLIC CONTESTS
Jesko
via web
OPEN LETTER TO THE COMMUNTY
Dear all !
We got the impression that there have been many people INVITED just for rating to support SINGLE IDEAS in the Betacup-Contest. Due to the official rules that´s fine as long as it´s fair rating.
We would like to invite all of you to have a quick discussion on
that.
We would love if we can manage to have a fair competition and a
good contact even if there are a lot of competitors in this
contest.
Even if everyone wants to win for sure, our shared goal is to build a communty with fair & clear rules and a great working atmosphere. And regarding the betacup contest: to reduce waist and support a greener direction for our wonderful planet. Let´s do not forget this.
We are a fast growing community, integrating different cultures
and ways of thinking.
Let´s choose how we want to work together and build a good
atmosphere.
Sure, for the official contest rules its fine to invite people to an open contest, as long as their rating is fair. What means "fair" to us exacty? This is the first time jovoto is trying this method, and i think we need at least some quick rules for this. Do we really need public rating ? Does it help the cmmunity spirit ?
We think the winning idea should not be chosen by "WHO INVITES
MOST FRIENDS FOR RATING ?".
Or "Who makes best PR on top of his work?"
Do you have any suggestions how we are going to deal with it ? Should we add some rules for new rating members? Are single or significant support ratings deleted by the monitoring ? Is there a sigificant difference ,when you look at the rating without new rating members, dear jovoto team ? Or is it just our personal impression ?
Do we have to invite all our friends or business contacts for rating now as well ?
It would be great if we have a quick and fruitful discussion and solution here with the whole community. To find quick answers or suggestions and clear official rules to be decided and chosen by the jovoto team.
How would we like to work here ?
yours Jan & Jesko
Showing the most recent page of comments. View the first page
Comments are currently closed for this discussion. You can start a new one.
62 Posted by mail on 07 Jun, 2010 12:07 PM
I agree with KLAR. For me jovoto is a public WORKING plattform and not a public VOTING plattform.
63 Posted by Janne on 07 Jun, 2010 12:46 PM
@bunterberg When you take it as given that a public contest encourages contributors to invite their friends, it seems that you actually want us to invite our friends as well... This might be okay for some fun sites but not for a serious site that wants to sell (!) design ideas to business (!) partners imo. I agree with voto here.
64 Posted by Robert Hosking on 07 Jun, 2010 01:27 PM
All I can say is it's strange to see my submission go through such wild extremes of position in the ratings. At one time I was in position 16 and at others as low as 60. Yesterday I was at 24 and today 42. It's seems to go all over the place. Makes me wonder how the rating system is set up. You can tell the top positions are really working hard to stay there.
65 Posted by kaptn on 07 Jun, 2010 01:31 PM
Hi,
I wrote it before. Why does not count the "Karma-Index"?
E.g. if someone is new and has a low karmaindex than the vote will not count the same as someone with a high index. So everybody has to work on this. By the way I have 5 brothers and sisters, 13 nieces and nephews,..... ;-(
K
66 Posted by Ellen Sandbeck on 07 Jun, 2010 01:35 PM
I have been quite dismayed at what has been going on during the "rating only" period of the Betacup Contest. The rankings have been bouncing around like ping pong balls, with no apparent relationship to the actual merit of the entries. If anything, the rankings are getting more unfair than before, as more and more contestants enlist their friends and relatives to help skew the contest, perhaps by giving the leading entries a very low vote.
I have decided to refrain from changing my votes during this period because it would be much too easy for me to fall into the trap of casting a "revenge vote" or a "strategic vote," and I don't want to do that. I have voted on a couple of entries that I hadn't voted on before, but now I'm done.
I will not be entering any more contests on Jovoto until the voting method is changed. I do not want to enter popularity contests; I want to enter idea contests.
67 Posted by mail on 07 Jun, 2010 01:55 PM
@kaptn: I think, linking vote to Karma isn't a good idea. I don't like jovoto's karma-system in general because as far as i can see, it is basically about quantity and not quality. Linking Karma to voting creates a kind of hirarchy which i don't want. So if someone is allowed to vote every single vote should be treated equal.
@Ellen Sandbeck: You are right!
68 Posted by Frieda on 07 Jun, 2010 02:56 PM
Hey all,
thanks a again for your comments and suggestions.
@KLAR, voto, Ellen: Let me stress one thing first: there are two contest types on jovoto, Public Contest and Private Contests. Public Contests are designed for a great number of participants, they are aimed at creating a great number of ideas AND at attracting the attention of a great number of people. In the case of the Betacup this is only reasonable: getting the word around is already a part of creating more attention for the problem with throw-away paper-cups. Creating awareness is a necessary step in reducing paper-cup waste - and this contest is partly aimed at doing so. So yes, the more people join in and comment and rate, the better.
On the other hand, Private Contest are indeed aimed at quite other objectives: professional creatives working on challenging tasks with proper prize money for the best ideas. Awareness rising and hence the attention of as much people as possible is NOT a part of Private Contests.
If you participate in a Public Contest you should be aware of the aims of the contest so as to avoid misunderstandings and disappointments. Please find information in the FAQs http://support.jovoto.com/faqs/contests/what-are-public-contests-an...
Private Contests will stay what they are: a closed environment for in-depth work and professional work. Only community members that prooved their skills AND good community behaviour (fair ratings etc.) are invited to join Private Contests.
Hence, dear KLAR, voto and Ellen, please don't judge on Public Contests with the criteria you're applying to Private Contests!
Concerning the rating system:
- the bouncing of ideas appears because people rate the ideas. This is actually a good sign and I don't understand why it's seen as a negative thing. An idea with 120 ratings obviously isn't bouncing around that much because it has a stabil foundation. An idea with only 15 ratings however is much more affected by each additional rating - hence it bounces around. Please, explain to me why this is bad? Also, when I look at the top ranks I see a lot of well-known ideas that are there since quite some time.
there is a proper protection against people trying to manipulte the system. We have strong algorithms at work that alarm the community management whenever something seems fishy. We confront "wrongdoers", inform them about appropriate rating behaviour, and if we see that they don't react on our querie to change their rating behaviour accordingly, we set their rating status on "false".
it is part of the intention of a Public Contest that community members invite friends and as long as these friends don't infringe on our rating rules (eg basing other ideas, only rating for one idea; find more info here http://support.jovoto.com/faqs/rating and http://www.jovoto.com/blog/de/kreative/gut-zu-wissen/votingsystem/) we have no reason to see this as something fishy. Again: this is however NOT somethinh we'd want in Private Contests.
I really hope that this made some things clearer to all of you. We're working on communicating Public and Private Contests better. I think major parts of the disgruntlement of some community members here is based on simple misdunderstandings. We do understand that clearer communication from our side is needed to not let any confusion arise in the future.
Best
Frieda
69 Posted by Ellen Sandbeck on 07 Jun, 2010 04:03 PM
Hi Frieda,
I understand what you are saying, but the fact of the matter is that many, many people are viewing the rating system with alarm, and if enough people view it that way, eventually the only people who may be willing to enter Jovoto contests are those who feel that they can win a popularity contest. I truly believe that the rating system needs to be radically altered so that exactly how one can game the system and win the contest is not so immediately obvious. Whether or not your algorithms can actually catch and stop cheaters is not really the point. When people watch the rankings swinging wildly, they are going to worry about it, and no amount of information about algorithms is going to be enough to stop the speculation about the fairness of the system.
Ellen
70 Posted by Shaun [Mutopo] on 07 Jun, 2010 04:57 PM
hello everyone.
I know the jovoto team continues to find ways to refine and improve the
rating system, but wanted to emphasize some critical points from the
perspective of thebetacup team
+ community prizes - voting is appealing because everyone has a say. there
are attempts made to limit gaming of the system but as people have pointed
out, it can be a popularity contest. are the popular ideas the best, it's
possible, but then again, they might not be.
so there are 2 other opportunities to rate and select
+ the jury - I can tell you this has been very very hard. the breadth of
ideas and their likely impact, viability, feasibility and impact on the
coffee drinking experience are not easy to judge. we've had a number of
debates and it confirms that even without gaming, rating is a very hard
process in this contest. but, we do believe the jury will bring a different
perspective and will offer another view of "the best" result.
+ starbucks - finally and importantly, starbucks will be assessing ideas to
determine if there ideas they can license and take to market. we expect that
is in this case, the results might differ from both the community and the
jury because they may focus on different areas like whether or not they have
tried ideas, how they might be licensed, investment requirements, etc.
we'll continue to monitor the ratings and work with the jovoto team. but I
hope this breakdown gives a sense of how we are trying to make the process
as fair as possible.
workplay with us
http://www.mutopo.com
http://www.thebetacup.com
@shaunabe +1 (917) 512-5571
sent from the future
71 Posted by bunterberg on 07 Jun, 2010 05:04 PM
@Frieda: can't be said better
@Shaun: totally agreed but no need to publish your mobile phone number ;)
72 Posted by Carey Babyak on 07 Jun, 2010 05:07 PM
I agree with Ellen. The perception of the ranking system is very very poor. It doesn't matter how you explain it, from someone who has submitted ratings and received ratings the system is inadequate. It is the equivalent of learning everyone's salary in the company where you work. You already knew how hard people work in comparison to yourself, but once you have a salary figure to accompany this knowledge the work to money earned ratio seems grossly disproportionate. Why should someone who works half as much make twice as much money. The same is true for the ratings. There are plenty of poor submissions with high ratings. Some of the submissions rely on borrowed information from previous submissions and add to tthem without credit. Some submissions have high ratings simply for their pretty pictures and not for the substance of their ideas.
Betacup will be my last interaction with Jovoto until a better platform is developed. The return on investment of my time is zero. My time is best spent elsewhere.
cbabyak
73 Posted by Bunterberg on 07 Jun, 2010 05:22 PM
@Carey of course there are various motivational patterns for people to attend. Extrinsic motivations cover earning prize money, selling ideas or building references and visibility or network to improve future business but also intrinsic motivations such as making meaning, socializing and learning or simply having fun can be considered reasons to participate.
Your answer tells me that you've maybe missed a lot of information within this discussion thread and it may worth a read
74 Posted by Charles on 07 Jun, 2010 06:52 PM
More money more problems. ::::shrugs:::::
@cbabyak imagine finally gaining rate and dropping in at a cool #7 and in < 2 hour at #50 something.- I saw at that point that I was taking this thing waaaay too serious which striped the fun out of it. We all know the ratings are jacked - moreover there people who submitted day one and some who submitted day 30 - who do you think will have the most rating? Even though the guy who submitted on day 30 his idea is much more superior....
Interesting enough my rating are 99.9% community driven - have invited no friends, no family or co-workers to rate. My strategy all alone has been to appeal to Jury and Sponsor not - why aim low when it takes the same amount of energy to aim high. Moreover I am persuaded that the jury and sponsor with be more objectionable - fairer.
My take is have fun aim high and I look forward to the impending changes to make public voting fairer. Jovoto if you got your ears on - "you have a good thing here don't let ill-gotten gainers ruin it for you/us - Like I always say - "I might can't kick their ares but I sure in hell can out create them - Get on your game Jovoto!
⇧Stay on Top⇧
75 Posted by Charles on 07 Jun, 2010 07:02 PM
Well - what do you know - UPS just delivered my new EVO - It's absolutely beautiful! I am in EVOL (i.e. LOVE) If you guys don't hear from me for a couple of days I will be playing with my EVOL = )
⇧Stay Up⇧
76 Posted by Janne on 08 Jun, 2010 06:47 AM
Dear jovoto,
yesterday I did a little experiment. Since you definitely encouraged us to invite our friends, I followed your advice and invited my husband and two daughters (14 and 17 yrs old) to this site. Of course I told them to not only rate my entry and furthermore to rate fair. What happened was the following:
My DH willingly rated me but as he didn´t have much time, asked me to do the rating for him. FAIR?
My younger DD asked me to help her with rating as she was confused with all those entries and besides her English isn´t very good. Of course I helped her. FAIR?
My eldest DD fortunately didn´t need any help. After asking whether she really rated fair, she answered: "Sure. I just looked which entries you liked and which you dedn´t and rated accordingly." FAIR?
Judge for yourself. My own conclusion is clear. Friends will not enter this site because they are interested to read 430 more or less longish concepts on a subject they are not really interested in. They enter this site to do their friend a favour. And since their friend told them so, they halfheartedly rate some more entries just to make it look legal.
I ask you again: Is this what you call fair rating? My personal answer is: It isn´t! and I probably won´t partake in an open jovoto contest again - that is neiter contrubuting any idea nor comment nor rating - as long as you don´t change the rating system. I sincerely ask you to think it over, unless you want to loose motivated and creative members.
Yours, Janne
77 Posted by Janne on 08 Jun, 2010 07:12 AM
...After I had written my post above, I made an interesting discovery:
Two people I don´t know "love"my idea and after checking who they are, I saw that they are both from the same country and that both their rating behaviour has been questioned. Now here is the perfect example of what happens, when you kindly ask unfair raters to do some more rating: They pick whatever entry (I had written a positive comment on an idea from a person of their country - so this might be the reason they choose me) and give it 10* as well...
Actually I should be happy - I am on #6 now. But I am not. I feel really annoyed and this whole rating thing spoiled my day.
Very upset, Janne
Support Staff 78 Posted by Nadine Freischlad on 08 Jun, 2010 01:02 PM
@Cbabyak @Charles @Janne yes, we are here and listening. It probably does not make sense to keep repeating the same arguments, Frieda and bunterberg have wrapped it up nicely. Since this thread is getting long and increasingly difficult to read, we decided to write up an official reaction to this entire thread. Give us time to take in your advice and the good suggestions that have come up - I will let you know.
Nadine Freischlad closed this discussion on 08 Jun, 2010 01:02 PM.