×
  • remind me tomorrow
  • remind me next week
  • never remind me
Subscribe to the ANN Newsletter • Wake up every Sunday to a curated list of ANN's most interesting posts of the week. read more

Forum - View topic
Haruhi's declining rating...


Goto page 1, 2  Next

Anime News Network Forum Index -> Site-related -> Encyclopedia
View previous topic :: View next topic  
Author Message
DonQuigleone



Joined: 31 May 2007
Posts: 154
Location: Dublin, Ireland
PostPosted: Fri Jul 13, 2007 7:41 am Reply with quote
This isn't so much a discussion of the series but I have noticed it has gone from being number 3 a few weeks back to number 9 now, I wouldn't think much of this but the weird thing is that 185 people have rated worst ever while only small numbers rated it at awful, bad, weak, not really good, so so and decent, in fact the only very good, excellent and masterpiece received higher votes, doesn't this sound like voting to change something's position in the ratings rather than give your real opinion, which I was under the impression was against the rules...
Back to top
View user's profile Send private message My Anime
dormcat
Encyclopedia Editor


Joined: 08 Dec 2003
Posts: 9902
Location: New Taipei City, Taiwan, ROC
PostPosted: Fri Jul 13, 2007 9:09 am Reply with quote
It's called rating spamming and we are aware of it. Similar issues have been discovered over two years ago.

Unfortunately, with the exception of evident ones e.g. creating multiple accounts to rate, it is virtually impossible to stop people from voting "worst ever" to some of the most popular series, for every user has the right to rate however the score with whatever reason. Checking and comparing the IP addresses of those 185 users would be time-consuming and ridiculous.

There is an algorithm to minimize the effect of spamming (and because of it you see the same number of votes does not necessary mean the same length of rainbow bars), but I don't know how it works (only Dan knows), and I wouldn't tell anyone even if I do know.
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger MSN Messenger ICQ Number My Anime My Manga
darksideoftheanime



Joined: 21 Jun 2007
Posts: 155
PostPosted: Fri Jul 13, 2007 9:55 am Reply with quote
dormcat wrote:
Unfortunately, with the exception of evident ones e.g. creating multiple accounts to rate, it is virtually impossible to stop people from voting "worst ever" to some of the most popular series, for every user has the right to rate however the score with whatever reason. Checking and comparing the IP addresses of those 185 users would be time-consuming and ridiculous.


I don't want to sound to mean but I think you should do anything that you can to make sure votes aren't being tampered with. If this means checking IP addresses go right ahead. Sure some people could use IP jumping programs but at lease it's a try.
Back to top
View user's profile Send private message
DonQuigleone



Joined: 31 May 2007
Posts: 154
Location: Dublin, Ireland
PostPosted: Fri Jul 13, 2007 12:06 pm Reply with quote
Ah so those bars represent how it's actually valued eh? It's not so bad then as the bars are not that long, also the same thing is happening with death note too
Back to top
View user's profile Send private message My Anime
cookie
Former ANN Editor in Chief


Joined: 02 Jan 2002
Posts: 2460
Location: Do not contact me for support.
PostPosted: Fri Jul 13, 2007 12:56 pm Reply with quote
darksideoftheanime wrote:
I don't want to sound to mean but I think you should do anything that you can to make sure votes aren't being tampered with. If this means checking IP addresses go right ahead. Sure some people could use IP jumping programs but at lease it's a try.


Sometimes it's easy to catch voting tampering because of the method by which ballot box stuffing is done. Sometimes it's not so easy. And sometimes it's actually genuine voting. Who are we to say "The guy over there who likes Starblazers and hates Haruhi is voting the wrong way"?
Back to top
View user's profile Send private message Visit poster's website My Anime My Manga
Tempest
I Run this place.
ANN Publisher


Joined: 29 Dec 2001
Posts: 10454
Location: Do not message me for support.
PostPosted: Fri Jul 13, 2007 3:49 pm Reply with quote
A few things worth noting:

1) It's common for very hyped titles to fall in the rank after a while.

2) Many people tend to vote towards the extreme, voting "best ever" and "worst ever" exclusively. This is why, for many titles, "worst ever" is longer than "awful." People who regularly vote at the extremes have very little significance in the ranking, so accounts that are created for the purposes of ballot stuffing don't really affect much.

3) Some of the people who voted "worst ever" for dub voted much higher for the subbed version. Only their higher vote counts towards the ranking.

4) Yes there is some ballot stuffing going on, but keep in mind there is probably positive ballot stuffing going on for Haruhi as well, also negative, and positive ballot stuffing for the titles above and below Haruhi on the ranking. What little effect it may have more or less cancels out.

5) Haruhi was just released on DVD, so it's to be expected that the # of votes would increase significantly in recent weeks.

6) Compared to some of the titles above Haruhi on the ranking, the number of people who voted "good" and "decent" has done more to bring Haruhi down in the ranking than the people who voted "Worst ever." Many of these people are the people referred to in #5, and they clearly aren't ballot stuffer, otherwise they would have voted one of the extremes.

All these issues combined make me feel very comfortable with the conclusion that there is nothing wrong with the ranking system. Certainly people try to abuse it, but they fail to have any effect.

I'd also like to add that any type of automated IP based ballot monitoring would do more harm than good. 500 people behind a proxy server or NAT, all voting legitimately, could have the same IP address. Some proxy servers forward the client IP, however NAT does no such thing.

-t
Back to top
View user's profile Send private message Send e-mail My Anime My Manga
Fallout2man



Joined: 27 Jun 2007
Posts: 274
Location: San Diego, CA
PostPosted: Fri Jul 13, 2007 5:36 pm Reply with quote
dormcat wrote:
It's called rating spamming and we are aware of it. Similar issues have been discovered over two years ago.

Unfortunately, with the exception of evident ones e.g. creating multiple accounts to rate, it is virtually impossible to stop people from voting "worst ever" to some of the most popular series, for every user has the right to rate however the score with whatever reason. Checking and comparing the IP addresses of those 185 users would be time-consuming and ridiculous.

There is an algorithm to minimize the effect of spamming (and because of it you see the same number of votes does not necessary mean the same length of rainbow bars), but I don't know how it works (only Dan knows), and I wouldn't tell anyone even if I do know.


Pseudo-SQL: change to fit your DB schema
Code:
SELECT sum(ip_address) AS vote_count,ip_address FROM `vote` GROUP BY `ip_address` WHERE anime_id=6430 ORDER BY `vote_count` DESC


That'd be a pretty fast way to check, if you know someone with direct access to the db.
Back to top
View user's profile Send private message AIM Address Yahoo Messenger MSN Messenger ICQ Number My Anime My Manga
Tempest
I Run this place.
ANN Publisher


Joined: 29 Dec 2001
Posts: 10454
Location: Do not message me for support.
PostPosted: Fri Jul 13, 2007 5:55 pm Reply with quote
Fallout2man wrote:

That'd be a pretty fast way to check, if you know someone with direct access to the db.


Thanks, but we're aware of the required sql code, but it's not feasible for us to manually check 6000+ titles, nor is there any point.

Although this issue was brought up for Haruhi, it's actually an issue for many titles. But as I explained above, it's more of a non-issue than actually issue. The system is such that the vote spam has a completely negligible effect. I'd be willing to bet that if all the spam was removed from the ency, Haruhi's rank wouldn't change.

-t
Back to top
View user's profile Send private message Send e-mail My Anime My Manga
DonQuigleone



Joined: 31 May 2007
Posts: 154
Location: Dublin, Ireland
PostPosted: Sat Jul 14, 2007 12:00 pm Reply with quote
I think you're right tempest, it probably is the recent dvd release and all... Anyway While we're on the topic one thing that would be easy to watch for are accounts with few votes and only votes at the extremes
Back to top
View user's profile Send private message My Anime
Tempest
I Run this place.
ANN Publisher


Joined: 29 Dec 2001
Posts: 10454
Location: Do not message me for support.
PostPosted: Sat Jul 14, 2007 2:06 pm Reply with quote
DonQuigleone wrote:
I think you're right tempest, it probably is the recent dvd release and all... Anyway While we're on the topic one thing that would be easy to watch for are accounts with few votes and only votes at the extremes


As previously stated, the system does "watch" those people. Their votes are worth less.

-t
Back to top
View user's profile Send private message Send e-mail My Anime My Manga
ikillchicken



Joined: 12 Feb 2007
Posts: 7272
Location: Vancouver
PostPosted: Sat Jul 14, 2007 11:19 pm Reply with quote
Honestly, Youre best off to just ignore these Ratings. Even without people manipulating the ratings its not going be all that close to accurate.

If you want to know how good something is, just go read a review. Theres tons of em out there. They'll be so much more ballanced than your average "OMG it was teh best/worst Anime ever!!!!" fan reaction.
Back to top
View user's profile Send private message My Anime
DonQuigleone



Joined: 31 May 2007
Posts: 154
Location: Dublin, Ireland
PostPosted: Sun Jul 15, 2007 4:23 pm Reply with quote
I dunno, the stuff at the top is very much sure fire good viewing (I make an exception for Kanon, but that might just be me Razz)
Back to top
View user's profile Send private message My Anime
Zalis116
Moderator


Joined: 31 Mar 2005
Posts: 6897
Location: Kazune City
PostPosted: Sun Jul 15, 2007 6:01 pm Reply with quote
DonQuigleone wrote:
I dunno, the stuff at the top is very much sure fire good viewing (I make an exception for Kanon, but that might just be me Razz)
The ratings are good as "ballpark estimates," which is why upper-echelon series are of good quality. Is Haruhi better than Death Note or Mushi-Shi? Maybe, maybe not, but the ratings indicate that it's in the same "class" as those two, and is at least better than series like this one. So that's why I don't worry about "What's #1? What's #2?" that much.
Back to top
View user's profile Send private message My Anime My Manga
abunai
Old Regular


Joined: 05 Mar 2004
Posts: 5463
Location: 露命
PostPosted: Sun Jul 15, 2007 6:19 pm Reply with quote
Zalis116 wrote:
So that's why I don't worry about "What's #1? What's #2?" that much.

What's on second.

- abunai
Back to top
View user's profile Send private message Visit poster's website My Anime My Manga
woelfie
Encyclopedia Editor


Joined: 02 Jan 2005
Posts: 380
Location: Belgium
PostPosted: Tue Jul 17, 2007 2:17 am Reply with quote
Zalis116 wrote:
Is Haruhi better than Death Note or Mushi-Shi? Maybe, maybe not, but the ratings indicate that it's in the same "class" as those two, and is at least better than series like this one. So that's why I don't worry about "What's #1? What's #2?" that much.

Indeed, why bother if Haruhi is ranked 1st, 10th or 50th ? Open the details in the user rating section, and you'll see that all ratings (median, arithmetic mean, weighted mean and bayesian estimate) state 'Excellent'. The rest is of minor importance.
Back to top
View user's profile Send private message Visit poster's website MSN Messenger ICQ Number My Anime My Manga
Display posts from previous:   
Reply to topic    Anime News Network Forum Index -> Site-related -> Encyclopedia All times are GMT - 5 Hours
Goto page 1, 2  Next
Page 1 of 2

 


Powered by phpBB © 2001, 2005 phpBB Group