Donor Rank isn't accurate

Moderators: Site Moderators, FAHC Science Team

Zagen30
Posts: 823
Joined: Tue Mar 25, 2008 12:45 am
Hardware configuration: Core i7 3770K @3.5 GHz (not folding), 8 GB DDR3 @2133 MHz, 2xGTX 780 @1215 MHz, Windows 7 Pro 64-bit running 7.3.6 w/ 1xSMP, 2xGPU

4P E5-4650 @3.1 GHz, 64 GB DDR3 @1333MHz, Ubuntu Desktop 13.10 64-bit

Re: Donor Rank isn't accurate

Post by Zagen30 »

Even the non-combined stats don't seem right http://fah-web.stanford.edu/cgi-bin/mai ... me=Zagen30. They claim that my current name/team combo is ranked 488 overall, but I just downloaded a copy of the user summary, opened it in Excel, and my rank is 999 (counting anonymous and PS3 as valid entries).
Image
VijayPande
Pande Group Member
Posts: 2058
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: Donor Rank isn't accurate

Post by VijayPande »

We've looked into this and it has to do with how we handle the combinations of donor names and passkeys. Right now, the flat file stats (which is what the 3rd party sites use) combine all passkeys for a single donor name into a single combined line. This is slow (mysql has to do lots of adds), but is ok for the flat files since it only has to be done rarely. The FAH web site just does an sql count(*) which is fast, but it can't combine accounts with passkeys. The early fix was to ignore accounts with passkeys (which is what the web site team did), but that's now a problem as there's lots of accounts with passkeys.

We're investigating what would be the best fix for this. I'm leaning on changing the flat files so each {donorname,passkey} combination gets its own line in the flat files. We would of course obscure the passkey value in the public stats.
Prof. Vijay Pande, PhD
Departments of Chemistry, Structural Biology, and Computer Science
Chair, Biophysics
Director, Folding@home Distributed Computing Project
Stanford University
klasseng
Posts: 125
Joined: Thu Dec 27, 2007 6:08 am
Hardware configuration: System #1, Quad GPU:
Motherboard: Asus Rampage IV Extreme
CPU: 6 Core Intel i7 (3930K)
GPU: 4 X NVIDIA GForce GTS 450
OS: WIndows 7 Home Premium, 64-bit
RAM: 16GB

System #2:
MacPro 2,1 (Early 2007)
Dual Quad-Core Intel Xeon 3GHz (X5365)
9GB Memory
OS: Mac OS X 10.7.5
GPU: N/A
Location: Canada

Re: Donor Rank isn't accurate

Post by klasseng »

Now almost 4 months later:
Donor Rank: 91
Daily User Summary: 339
EOC Ranking: 337

Only getting worse
peace,
Grant
VijayPande
Pande Group Member
Posts: 2058
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: Donor Rank isn't accurate

Post by VijayPande »

Thanks for bumping this one. There hasn't been a huge demand for this amongst the donors (eg your the first to comment on it in several months), so it was pushed to a lower priority. Is this something that bothers a lot of other people as well? If so, I'll re-route resources away from other on-going code development to finalize this one.
Prof. Vijay Pande, PhD
Departments of Chemistry, Structural Biology, and Computer Science
Chair, Biophysics
Director, Folding@home Distributed Computing Project
Stanford University
goodyca
Posts: 187
Joined: Sun Dec 02, 2007 12:36 pm

Re: Donor Rank isn't accurate

Post by goodyca »

I would also like for the rankings to agree. My ranking are:

Donor Rank: 1026
daily user summary: 2135
derrickmcc
Posts: 221
Joined: Fri Jul 24, 2009 12:30 am
Hardware configuration: 2 x GTX 460 (825/1600/1650)
AMD Athlon II X2 250 3.0Ghz
Kingston 2Gb DDR2 1066 Mhz
MSI K9A2 Platinum
Western Digital 500Gb Sata II
LiteOn DVD
Coolermaster 900W UCP
Antec 902
Windows XP SP3
Location: Malvern, UK

Re: Donor Rank isn't accurate

Post by derrickmcc »

I would appreciate this being fixed.
Donor Rank: 532
EOC rank: 1211
Image
KMac
Posts: 31
Joined: Thu Feb 17, 2011 6:50 pm

Re: Donor Rank isn't accurate

Post by KMac »

While I don't want resources reallocated away from other projects, this inaccuracy should be addressed eventually. Inaccuracies and/or discrepancies do not lend credibility to the project.
As a temporary work-around, couldn't you just stop posting the inaccurate Donor Rank until the issue can be resolved?
Donor Rank 71
EOC Rank 275
VijayPande
Pande Group Member
Posts: 2058
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: Donor Rank isn't accurate

Post by VijayPande »

KMac wrote:Inaccuracies and/or discrepancies do not lend credibility to the project.
As a temporary work-around, couldn't you just stop posting the inaccurate Donor Rank until the issue can be resolved?
Donor Rank 71
EOC Rank 275

ok, while I see your point about the discrepancy being a bad thing, I think there's a misunderstanding here. The Stanford web pages are correct and the 3rd party stats pages are not, due to incomplete information given to the 3rd party stats suppliers (a more complete discussion is in an earlier post).

However, they cannot fix the pages without more information, hence the new download link we're working on. I guess we could ask the 3rd party stats to stop putting a ranking but I don't think that helps here much.
Prof. Vijay Pande, PhD
Departments of Chemistry, Structural Biology, and Computer Science
Chair, Biophysics
Director, Folding@home Distributed Computing Project
Stanford University
EOC_Jason
Posts: 10
Joined: Wed Feb 10, 2010 3:15 pm
Location: USA
Contact:

Re: Donor Rank isn't accurate

Post by EOC_Jason »

Don't blame Stanford on this. For me (EOC Stats) I kind of have to guesstimate it since there is no "rank" per say in the flat file. The individuals are merely sorted by their points, so I just have a "row count" as it parses and that's how I put in their overall rank. This doesn't account for the merged users or also the time-lapse between updates so it's never going to be 100% identical. I made the EOC stats mostly for people within their own teams to track progress, and also team vs team... The big thing is being able to handle multiple accounts that appear to have the same username on the same team, that's where the problem is... I say "appear" because they aren't actually the same account, stanford just does some filtering before the flat file is generated which makes it that way (to prevent personal data from being exposed).
patonb
Posts: 348
Joined: Thu Oct 23, 2008 2:42 am
Hardware configuration: WooHoo= SR-2 -- L5639 @ ?? -- Evga 560ti FPB -- 12Gig Corsair XMS3 -- Corsair 1050hx -- Blackhawk Ultra

Foldie = @3.2Ghz -- Noctua NH-U12 -- BFG GTX 260-216 -- 6Gig OCZ Gold -- x58a-ud3r -- 6Gig OCZ Gold -- hx520

Re: Donor Rank isn't accurate

Post by patonb »

Thanks Jason for stepping up and setting the record straight.

EOC is perfect for what its intended use was, team challanges.
WooHoo = L5639 @ 3.3Ghz Evga SR-2 6x2gb Corsair XMS3 CM 212+ Corsair 1050hx Blackhawk Ultra EVGA 560ti

Foldie = i7 950@ 4.0Ghz x58a-ud3r 216-216 @ 850/2000 3x2gb OCZ Gold NH-u12 Heatsink Corsair hx520 Antec 900
Zagen30
Posts: 823
Joined: Tue Mar 25, 2008 12:45 am
Hardware configuration: Core i7 3770K @3.5 GHz (not folding), 8 GB DDR3 @2133 MHz, 2xGTX 780 @1215 MHz, Windows 7 Pro 64-bit running 7.3.6 w/ 1xSMP, 2xGPU

4P E5-4650 @3.1 GHz, 64 GB DDR3 @1333MHz, Ubuntu Desktop 13.10 64-bit

Re: Donor Rank isn't accurate

Post by Zagen30 »

The changes seem to have gone into effect. My combined ranking is more believable- before it was 282, which seemed unlikely since my main name/team ranking was only in the 900's, while now it's 687.

I did notice something odd, however. My current name/team ranking is 938 on EOC and 938 if I copy/paste the donor list into Excel (leaving out anonymous and PS3), but on the stats page it's 858. I may be wrong, but aren't the individual name/team rankings still calculated assuming that every name/team is a different entity? Why is there such a large discrepancy here?
Image
VijayPande
Pande Group Member
Posts: 2058
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: Donor Rank isn't accurate

Post by VijayPande »

We're still working out some issues and are in contact with 3rd party stats developers to try to get the agreement perfect.
Prof. Vijay Pande, PhD
Departments of Chemistry, Structural Biology, and Computer Science
Chair, Biophysics
Director, Folding@home Distributed Computing Project
Stanford University
shunter
Posts: 84
Joined: Sun Apr 06, 2008 8:22 am
Location: Hertfordshire, United Kingdom

Re: Donor Rank isn't accurate

Post by shunter »

Well that's a bit of a shock - always thought Stanford gave a better ranking than I deserved or that given by independent stats sites but just dropped from 186 to 484 in a day
Image
kiore
Posts: 931
Joined: Fri Jan 16, 2009 5:45 pm
Location: USA

Re: Donor Rank isn't accurate

Post by kiore »

I would hate to see this issue remove resources from more important things, like moving the V7 out of Beta etc. Really despite the differences in donor rank between the different the different stats systems, if you want to compare, just compare on the same system.
Mind you this discussion lead me to check my official rank .. 380th wow, that snuck up on me! But I have donated in several teams.. so a bit complex.
Image
i7 7800x RTX 3070 OS= win10. AMD 3700x RTX 2080ti OS= win10 .

Team page: http://www.rationalskepticism.org
VijayPande
Pande Group Member
Posts: 2058
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: Donor Rank isn't accurate

Post by VijayPande »

After looking at this more (and thanks to the posts in this thread for bringing this up), I agree that there is an issue here on our side. This stems from the fact that many people donated with and without a passkey (sometimes with multiple passkeys) and how our stats system calculates ranking on the fly. Clearly, the best way to solve this is to do a GROUP BY donorname. This will make the Stanford stats completely agree with the 3rd party (by grouping all of the passkeys for a given donorname into a single donorname), but this is a very slow SQL query. We do it when we make the 3rd party stats flat files, since we do that infrequently so that's not a problem.

What's the impact of not doing the GROUP BY? We can see that when we look at a good example: donors in the top 10 list on our main page (http://fah-web.stanford.edu/cgi-bin/mai ... =userstats). If you look at AtlasFolder (http://fah-web.stanford.edu/cgi-bin/mai ... tlasFolder), you'll see that he's contributed with multiple teams. With the GROUP BY, we get this right (if combining everything into one name is what we're going for). If we do the queries as we did above, we get the problem that people are mentioning above.

Anyway, that's where we are now. We're aware of this issue and we've been talking to 3rd party stats to see what's the best way to handle this in a way that works for them. So far, the leading possibility would be for us to add a rank column to the db and update it every time we do a stats update (we would do a GROUP BY and then just put the results into the db). That removes some possibilities for dynamic updating, but would resolve the issues we're talking about above.

In the mean time, what would donors prefer us to do: how about we remove ranking from our site to until we get things straightened out?
Prof. Vijay Pande, PhD
Departments of Chemistry, Structural Biology, and Computer Science
Chair, Biophysics
Director, Folding@home Distributed Computing Project
Stanford University
Post Reply