Comparative scientific productivity between clients

Moderators: Site Moderators, FAHC Science Team

Post Reply
7up1n3
Posts: 68
Joined: Sun Dec 02, 2007 2:55 am
Contact:

Comparative scientific productivity between clients

Post by 7up1n3 »

I first participated in the Folding@Home project back in 2002, and have watched with interest as new client options continue to be made available (most recently the GPU2 client) and existing clients optimized (eg v4, v5, v6 of the standard Window's client). Flash forward to today, and we now have five major client platforms:
  • Windows
  • Linux
  • MacOS
  • PS3
  • GPU
With subsets (graphical, text, screensaver) within each category as well the ability to choose SMP clients for multi-core / multi-proc platforms.

With all the different options available, with the options in different developmental stages (eg Windows SMP remains in rather rough beta stage after several years), I think it would be helpful to know how the clients measure up against each other in regard to their *scientific value*. While I'm assuming there isn't a lot of difference between the different OS platforms in the regard, it would make sense to me that there would be some variance between:
  • CPU specific standard vs SMP client productivity
  • CPU vs GPU vs PS3 client productivity
  • ATi vs Nvidia productivity
Why helpful? Because, while points and competition is pretty fun, the scientific value of our participation is (or can be with the right information) a key factor to consider when one is in a purchasing mode. And it may very well be that points aren't a very good indicator of the actual scientific value contributed across the different platforms. Armed with this information, users would be able to consider more than just the perceived point productivity when looking at adding new hardware to their 'Fold', perhaps focusing instead on the upgrade's scientific value.
Image
Rage3D Admin ~ The Fighting 300 ~ Team Rage3D Folding
Ivoshiee
Site Moderator
Posts: 822
Joined: Sun Dec 02, 2007 12:05 am
Location: Estonia

Re: Comparative scientific productivity between clients

Post by Ivoshiee »

Currently running and completed projects per client type is one benchmark as well.
gwildperson
Posts: 450
Joined: Tue Dec 04, 2007 8:36 pm

Re: Comparative scientific productivity between clients

Post by gwildperson »

Good questions. I'd like to see the answers, too.

It's too soon to know the answer to at least one of your questions: ATi vs Nvidia productivity.

The ATi client has been running long enough to have produced some genuine scientific results, but the Nvidia client is still running 100% test assignments so although they're useful to debug the software, they have produced zero new scientific results. It's reasonable to assume that the scientific value of the results will be equivalent for hardware that runs the same speed since the science code in FahCore_11 is identical, but that's still an assumption.

That brings up another question. How should the answers manage hardware differences? You didn't ask about the scientific differences between Intel vs. AMD CPUs. That's reasonable because they run identical software so the only differences are in the features of the specific hardware you're comparing. Presumably the same answer will apply to ATI vs Nvidia so it's not a fair question.

BTW, there are two distinctly different MacOS clients. Maybe you forgot about the old uniprocessor client which is very different than the new smp client for the Intel hardware.
7up1n3
Posts: 68
Joined: Sun Dec 02, 2007 2:55 am
Contact:

Re: Comparative scientific productivity between clients

Post by 7up1n3 »

Ivoshiee wrote:Currently running and completed projects per client type is one benchmark as well.
Wouldn't that be Indicative rather than definitive, in regard to actual scientific value?
gwildperson wrote:That brings up another question. How should the answers manage hardware differences? You didn't ask about the scientific differences between Intel vs. AMD CPUs. That's reasonable because they run identical software so the only differences are in the features of the specific hardware you're comparing. Presumably the same answer will apply to ATI vs Nvidia so it's not a fair question.
If there is a scientific difference between different CPU architectures (other than # of cores probably), the answers would certainly relevant to the question as well. I haven't Folded with a current AMD quad-core processor, so do not know how they compare clock for clock from a productivity basis.
gwildperson wrote:BTW, there are two distinctly different MacOS clients. Maybe you forgot about the old uniprocessor client which is very different than the new smp client for the Intel hardware.
Yeah, I gave a pretty rough lineup of the existing clients, so there certainly are some gaps. I did, however, differentiate between the SMP & uniprocessor clients, though my perspective is based on Windows/Linux clients, so its likely I'm not aware of all the differentiating factors within the MacOS client arsenal.
Image
Rage3D Admin ~ The Fighting 300 ~ Team Rage3D Folding
codysluder
Posts: 1024
Joined: Sun Dec 02, 2007 12:43 pm

Re: Comparative scientific productivity between clients

Post by codysluder »

One way of looking at it is flops. I presume many of you have seen this:
Subject: Highwater Mark

The Pande Group has repeatedly stated that flops are not an ideal measure of scientific results, so we have to use the information presented there more intelligently.

Perhaps the Pande Group can give us a science factor for each client. For example, they've said that the PS3 can't do everything that the uniprocessor client can do so let's guess that 2.5 PS3 FLOPS produces about the same amount of science as 1 FLOPS on Intel/AMD. (The 2.5 number is a sheer guess with no basis, but perhaps Dr.Pande can give a realistic number.) Wouldn't that make the data on the chart meaningful?
7im
Posts: 10189
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:

Re: Comparative scientific productivity between clients

Post by 7im »

You won't get the answers you are looking for, not in the last 5 years since I started folding, and not now. It's like asking a parent to pick their favorite child.

While one client might be more productive, it might be less relevant. Or a less productive client might be generating results for an important paper that is just about to be published. And like work units and PPD, the science changes from week to week, so today's answer would be different next week.

The only clear choice is points based. Points are mostly science based, but do factor in other criteria. Go where the points take you.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
VijayPande
Pande Group Member
Posts: 2058
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: Comparative scientific productivity between clients

Post by VijayPande »

I think 7im put it well. We try to balance the points based on both performance and scientific usefulness. We understand that donors will optimize for points, so our goal is to align points with the scientific usefulness (as well as we can, without making dramatic changes to the system too frequently).
Maverick21
Posts: 2
Joined: Wed Sep 03, 2008 2:23 pm

Re: Comparative scientific productivity between clients

Post by Maverick21 »

For some 'time I've been searching an answer to this question.

If I have understood correctly, points are an indicator of how important is in that moment that kind of work, is right?
So in this moment one can deduce that the project has a great need of GPU since they get a lot of points compared to CPU, but there is no doubt that a 5000 PPD GPU doesn't give a 10x "scientific work" compared to a 500 PPD CPU.

Regarding the PPD difference between the normal cpu client and the SMP can we say the same thing? They are different about the importance for the project but not about actual scientific work done?
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Comparative scientific productivity between clients

Post by bruce »

I think you're confusing points with points per day.

If project A does important work, we can assign it a value of 1 point.
If project B does the same type of work but it includes twice as much analysis, we can award it 2 points (You've done the same amount of important work if you complete two projects similar to project A.

If project C is much more important than project A but takes the same amount of time as project A, it should earn 2 points.

Now suppose Machine A runs 24x7 and it can finish 100 work assignments from project A or 50 work assignments from project B That machine earns 100 PPD. If it happens to be assigned project C that same machine might earn as much as 200 PPD, but with a mix of A, B, and C, it will earn more than 100 and less than 200 PPD.

Now suppose the GPU can complete 2000 work assignments from project A in a single day. That machine earns 2000 PPD. Since the GPU is unable to do work of type C, it will earn precisely 2000 PPD.

As a practical matter, the Pande Group might package a few hundred blocks of work of type A or B or C into a single package called a Work Unit.

Of course if the first machine can run project C, it can be earning any amount between

Does it begin to make more sense now?
chumbucket843
Posts: 10
Joined: Sun Sep 06, 2009 2:24 am

Re: Comparative scientific productivity between clients

Post by chumbucket843 »

well now its 2009. how is the nvidia gpu2 client doing? is bigadv doing well? ETA for GPU3?

one of the most gratifying things to see as a folding enthusiast is results. even a post that says its working is good to hear.
Flathead74
Posts: 266
Joined: Sun Dec 02, 2007 6:08 pm
Location: Central New York
Contact:

Re: Comparative scientific productivity between clients

Post by Flathead74 »

chumbucket843 wrote:well now its 2009. how is the nvidia gpu2 client doing? is bigadv doing well? ETA for GPU3?

one of the most gratifying things to see as a folding enthusiast is results. even a post that says its working is good to hear.
Prof. Vijay Pande's blog:
http://folding.typepad.com/news/
7im
Posts: 10189
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:

Re: Comparative scientific productivity between clients

Post by 7im »

It's working.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
toTOW
Site Moderator
Posts: 6296
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:

Re: Comparative scientific productivity between clients

Post by toTOW »

Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
chumbucket843
Posts: 10
Joined: Sun Sep 06, 2009 2:24 am

Re: Comparative scientific productivity between clients

Post by chumbucket843 »

7im wrote:It's working.
good to hear.
Post Reply