Donor Rank isn't accurate

Moderators: Site Moderators, PandeGroup

Re: Donor Rank isn't accurate

Postby VijayPande » Fri Mar 25, 2011 11:28 pm

We've looked into this and it has to do with how we handle the combinations of donor names and passkeys. Right now, the flat file stats (which is what the 3rd party sites use) combine all passkeys for a single donor name into a single combined line. This is slow (mysql has to do lots of adds), but is ok for the flat files since it only has to be done rarely. The FAH web site just does an sql count(*) which is fast, but it can't combine accounts with passkeys. The early fix was to ignore accounts with passkeys (which is what the web site team did), but that's now a problem as there's lots of accounts with passkeys.

We're investigating what would be the best fix for this. I'm leaning on changing the flat files so each {donorname,passkey} combination gets its own line in the flat files. We would of course obscure the passkey value in the public stats.
Prof. Vijay Pande, PhD
Departments of Chemistry, Structural Biology, and Computer Science
Chair, Biophysics
Director, Folding@home Distributed Computing Project
Stanford University
User avatar
VijayPande
Pande Group Member
 
Posts: 2651
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: Donor Rank isn't accurate

Postby klasseng » Mon Jun 20, 2011 6:25 am

Now almost 4 months later:
Donor Rank: 91
Daily User Summary: 339
EOC Ranking: 337

Only getting worse
peace,
Grant
User avatar
klasseng
 
Posts: 145
Joined: Thu Dec 27, 2007 6:08 am
Location: Canada

Re: Donor Rank isn't accurate

Postby VijayPande » Mon Jun 20, 2011 5:31 pm

Thanks for bumping this one. There hasn't been a huge demand for this amongst the donors (eg your the first to comment on it in several months), so it was pushed to a lower priority. Is this something that bothers a lot of other people as well? If so, I'll re-route resources away from other on-going code development to finalize this one.
User avatar
VijayPande
Pande Group Member
 
Posts: 2651
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: Donor Rank isn't accurate

Postby goodyca » Tue Jun 21, 2011 11:22 am

I would also like for the rankings to agree. My ranking are:

Donor Rank: 1026
daily user summary: 2135
goodyca
 
Posts: 152
Joined: Sun Dec 02, 2007 12:36 pm

Re: Donor Rank isn't accurate

Postby derrickmcc » Tue Jun 21, 2011 6:46 pm

I would appreciate this being fixed.
Donor Rank: 532
EOC rank: 1211
Image
User avatar
derrickmcc
 
Posts: 329
Joined: Fri Jul 24, 2009 12:30 am
Location: Malvern, UK

Re: Donor Rank isn't accurate

Postby KMac » Tue Jun 21, 2011 7:16 pm

While I don't want resources reallocated away from other projects, this inaccuracy should be addressed eventually. Inaccuracies and/or discrepancies do not lend credibility to the project.
As a temporary work-around, couldn't you just stop posting the inaccurate Donor Rank until the issue can be resolved?
Donor Rank 71
EOC Rank 275
KMac
 
Posts: 36
Joined: Thu Feb 17, 2011 6:50 pm

Re: Donor Rank isn't accurate

Postby VijayPande » Tue Jun 21, 2011 10:02 pm

KMac wrote:Inaccuracies and/or discrepancies do not lend credibility to the project.
As a temporary work-around, couldn't you just stop posting the inaccurate Donor Rank until the issue can be resolved?
Donor Rank 71
EOC Rank 275



ok, while I see your point about the discrepancy being a bad thing, I think there's a misunderstanding here. The Stanford web pages are correct and the 3rd party stats pages are not, due to incomplete information given to the 3rd party stats suppliers (a more complete discussion is in an earlier post).

However, they cannot fix the pages without more information, hence the new download link we're working on. I guess we could ask the 3rd party stats to stop putting a ranking but I don't think that helps here much.
User avatar
VijayPande
Pande Group Member
 
Posts: 2651
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: Donor Rank isn't accurate

Postby EOC_Jason » Tue Jun 21, 2011 10:05 pm

Don't blame Stanford on this. For me (EOC Stats) I kind of have to guesstimate it since there is no "rank" per say in the flat file. The individuals are merely sorted by their points, so I just have a "row count" as it parses and that's how I put in their overall rank. This doesn't account for the merged users or also the time-lapse between updates so it's never going to be 100% identical. I made the EOC stats mostly for people within their own teams to track progress, and also team vs team... The big thing is being able to handle multiple accounts that appear to have the same username on the same team, that's where the problem is... I say "appear" because they aren't actually the same account, stanford just does some filtering before the flat file is generated which makes it that way (to prevent personal data from being exposed).
EOC_Jason
 
Posts: 7
Joined: Wed Feb 10, 2010 3:15 pm
Location: USA

Re: Donor Rank isn't accurate

Postby patonb » Tue Jun 21, 2011 10:11 pm

Thanks Jason for stepping up and setting the record straight.

EOC is perfect for what its intended use was, team challanges.
WooHoo = L5639 @ 3.3Ghz Evga SR-2 6x2gb Corsair XMS3 CM 212+ Corsair 1050hx Blackhawk Ultra EVGA 560ti

Foldie = i7 950@ 4.0Ghz x58a-ud3r 216-216 @ 850/2000 3x2gb OCZ Gold NH-u12 Heatsink Corsair hx520 Antec 900
patonb
 
Posts: 943
Joined: Thu Oct 23, 2008 2:42 am

Re: Donor Rank isn't accurate

Postby Zagen30 » Wed Jun 22, 2011 7:13 pm

The changes seem to have gone into effect. My combined ranking is more believable- before it was 282, which seemed unlikely since my main name/team ranking was only in the 900's, while now it's 687.

I did notice something odd, however. My current name/team ranking is 938 on EOC and 938 if I copy/paste the donor list into Excel (leaving out anonymous and PS3), but on the stats page it's 858. I may be wrong, but aren't the individual name/team rankings still calculated assuming that every name/team is a different entity? Why is there such a large discrepancy here?
Image
Zagen30
 
Posts: 1589
Joined: Tue Mar 25, 2008 12:45 am

Re: Donor Rank isn't accurate

Postby VijayPande » Wed Jun 22, 2011 8:41 pm

We're still working out some issues and are in contact with 3rd party stats developers to try to get the agreement perfect.
User avatar
VijayPande
Pande Group Member
 
Posts: 2651
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: Donor Rank isn't accurate

Postby shunter » Wed Jun 22, 2011 10:54 pm

Well that's a bit of a shock - always thought Stanford gave a better ranking than I deserved or that given by independent stats sites but just dropped from 186 to 484 in a day
Image
shunter
 
Posts: 87
Joined: Sun Apr 06, 2008 8:22 am
Location: Hertfordshire, United Kingdom

Re: Donor Rank isn't accurate

Postby kiore » Fri Jun 24, 2011 9:09 pm

I would hate to see this issue remove resources from more important things, like moving the V7 out of Beta etc. Really despite the differences in donor rank between the different the different stats systems, if you want to compare, just compare on the same system.
Mind you this discussion lead me to check my official rank .. 380th wow, that snuck up on me! But I have donated in several teams.. so a bit complex.
Image

i7 7800x GTX 1080ti , OS= win7. AMD 3700x RTX 2080ti OS= win10
kiore
 
Posts: 996
Joined: Fri Jan 16, 2009 5:45 pm
Location: USA

Re: Donor Rank isn't accurate

Postby VijayPande » Sat Jun 25, 2011 12:39 am

After looking at this more (and thanks to the posts in this thread for bringing this up), I agree that there is an issue here on our side. This stems from the fact that many people donated with and without a passkey (sometimes with multiple passkeys) and how our stats system calculates ranking on the fly. Clearly, the best way to solve this is to do a GROUP BY donorname. This will make the Stanford stats completely agree with the 3rd party (by grouping all of the passkeys for a given donorname into a single donorname), but this is a very slow SQL query. We do it when we make the 3rd party stats flat files, since we do that infrequently so that's not a problem.

What's the impact of not doing the GROUP BY? We can see that when we look at a good example: donors in the top 10 list on our main page (http://fah-web.stanford.edu/cgi-bin/mai ... =userstats). If you look at AtlasFolder (http://fah-web.stanford.edu/cgi-bin/mai ... tlasFolder), you'll see that he's contributed with multiple teams. With the GROUP BY, we get this right (if combining everything into one name is what we're going for). If we do the queries as we did above, we get the problem that people are mentioning above.

Anyway, that's where we are now. We're aware of this issue and we've been talking to 3rd party stats to see what's the best way to handle this in a way that works for them. So far, the leading possibility would be for us to add a rank column to the db and update it every time we do a stats update (we would do a GROUP BY and then just put the results into the db). That removes some possibilities for dynamic updating, but would resolve the issues we're talking about above.

In the mean time, what would donors prefer us to do: how about we remove ranking from our site to until we get things straightened out?
User avatar
VijayPande
Pande Group Member
 
Posts: 2651
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: Donor Rank isn't accurate

Postby Pick2 » Sat Jun 25, 2011 2:12 am

Personally , Close is good enough for Me , save the effort for the science.
Pick2
 
Posts: 167
Joined: Fri Feb 13, 2009 12:38 pm
Location: USA

PreviousNext

Return to Discussions of General-FAH topics

Who is online

Users browsing this forum: No registered users and 3 guests

cron