Guidance for building a folding dedicated computer

A forum for discussing FAH-related hardware choices and info on actual products (not speculation).

Moderator: Site Moderators

Forum rules
Please read the forum rules before posting.

Re: Guidance for building a folding dedicated computer

Postby boristsybin » Fri Aug 18, 2017 10:12 am

QuintLeo wrote:AMD FX series are a lot lower cost for the same number of cores,
...
Ignore "hyperthreaded" cores - they do NOT work well to support folding GPUs, they MIGHT be worth a QUARTER of a real core.
...
In fact, ignore AMD

AMD FX supports only pci-e v2, that, in theory, may limit gpu productivity

i3 4130 with HT enabled is enough for three 1080ti, so HT does work well to support folding GPUs

amd gpus loads CPU less than nVidia`s :
with amd gpu, FahCore uses 100% per thread on task start and and drops to ~60% per thread on main time of task count,
with nVidia, FahCore uses 100% of CPU thread whole folding time.
Image
boristsybin
 
Posts: 45
Joined: Mon Jan 16, 2017 11:40 am
Location: Russia, Moscow

Re: Guidance for building a folding dedicated computer

Postby rwh202 » Fri Aug 18, 2017 4:53 pm

Leonardo wrote:
foldy wrote:RX Vega 64 makes 600-850k PPD so that is still reasonable too but uses more power.
Unless your electricity rates are very low, I think that performance is not good considering the power consumption of that GPU.

I think it's worth pointing out to those that are concerned by power use, that both manufacturers ship their cards past the optimal point in the power / performance curve.
Good results can be had by setting lower power targets than those that the cards ship with.
I have a rig with a pair of 1080 FE with enforced power limits of 130 W on them (down from 180 W). Performance is still good (650 - 700 k PPD per card) and just less than 300 W from the wall.
Overclocked 1080s might do 900 k PPD, but for double the power draw.
rwh202
 
Posts: 292
Joined: Mon Nov 15, 2010 8:51 pm
Location: South Coast, UK

Re: Guidance for building a folding dedicated computer

Postby foldy » Fri Aug 18, 2017 7:52 pm

GTX 1080 ti with 55% power limit 1000k PPD at 250 watts (incl. CPU) instead of 1200k PPD at 350 watts.
foldy
 
Posts: 942
Joined: Sat Dec 01, 2012 3:43 pm

Re: Guidance for building a folding dedicated computer

Postby Leonardo » Sat Aug 19, 2017 3:53 am

Thanks, Foldy. I've never tried adjusting power downward, at least not for Folding. I may just give that a try this weekend and see how it shakes out.
Image
User avatar
Leonardo
 
Posts: 655
Joined: Tue Dec 04, 2007 5:09 am
Location: Eagle River, Alaska

Re: Guidance for building a folding dedicated computer

Postby SteveWillis » Sat Aug 19, 2017 10:29 am

I have a couple of GPUs that run hot, around 90C that I reduce power on to cool them off and slow their fans. I don't like to run much over 80C and 75% fan.
For anyone running nvidia cards on Linux:

To display the current power
Code: Select all
nvidia-smi


To set the power for GPU 0
Code: Select all
sudo -n nvidia-smi -i 0 --persistence-mode=1
sudo -n nvidia-smi -i 0 --power-limit=115
Image
My thanks to my very indulgent wife
http://folding.extremeoverclocking.com/user_summary.php?s=&u=712804

3 AMD Linux rigs 3, 4, and 5 GPUs 7 X GTX 1080, 5 X GTX 1080 TI
SteveWillis
 
Posts: 227
Joined: Fri Apr 15, 2016 12:42 am

Re: Guidance for building a folding dedicated computer

Postby SombraGuerrero » Sat Sep 02, 2017 7:52 pm

So, in reading this thread, taking some new power readings on my rig, and doing some of my own quick calculations, I realize that folding with my CPU adds about 60-70 watts to my power load. For that extra load, I'm getting roughly about 43 - 77 PPD/watt/WU. This pales in comparison to my single GPU, which gives me about 140 - 300 PPD/watt/WU. So, this means I'm getting not even 1 PPD/watt on my CPU. From an energy cost perspective, it seems like I'd be better off not folding with my CPU. Is that a fair assessment, or are my numbers too wishy-washy?
SombraGuerrero
 
Posts: 98
Joined: Mon Mar 16, 2009 3:06 am

Re: Guidance for building a folding dedicated computer

Postby Leonardo » Sat Sep 02, 2017 11:44 pm

Fair assessment. That is exactly why so many of us don't bother with setting up CPU slots.
User avatar
Leonardo
 
Posts: 655
Joined: Tue Dec 04, 2007 5:09 am
Location: Eagle River, Alaska

Re: Guidance for building a folding dedicated computer

Postby SombraGuerrero » Sun Sep 03, 2017 3:27 am

I suppose there is an argument that the CPU cores obviously focus on different projects than the GPU cores, so I would hate to think that those get less attention, but I suppose there will always be plenty of other CPU folders out there.....
SombraGuerrero
 
Posts: 98
Joined: Mon Mar 16, 2009 3:06 am

Re: Guidance for building a folding dedicated computer

Postby bruce » Sun Sep 03, 2017 7:13 am

The "general rule" does not apply to everybody. I can see several possibles.

1) You have a Windows Gaming machine or the equivalent Linux machine with one or more powerful GPUs and a CPU that supports many threads.
> 1A) You pay for power.
> 1B) You live in a home or a school Dorm where somebody else pays the electric bill -- or you have a large enough roof-top solar system so that you rarely have an electric bill.
2) you have a laptop with a weak iGPU that can't handle FAH so the CPU is your only choice.
3) You run Mac OS-X (where FAH does not support GPUs) so again, the CPU is your only choice..

You guys in group 1A decide CPUs aren't practical.
Those in groups 1B, 2, and 3 run with their CPUs.

Let's not denigrate the contributions that others can make just because you happen have a powerful system. All contributions to science are valuable.
bruce
 
Posts: 21407
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Guidance for building a folding dedicated computer

Postby SombraGuerrero » Sun Sep 03, 2017 7:45 am

My feelings exactly, Bruce. I pay for electricity, but I wanted to run these numbers in the interest of awareness. Although it is an interesting exercise in conservation to run these types of numbers, I also feel that unless there is a grave need to cut consumption, it is sort've a slight to the CPU core project owners to ignore their work in the interest of "practicality."
SombraGuerrero
 
Posts: 98
Joined: Mon Mar 16, 2009 3:06 am

Re: Guidance for building a folding dedicated computer

Postby QuintLeo » Tue Sep 05, 2017 2:30 am

boristsybin wrote:
QuintLeo wrote:AMD FX series are a lot lower cost for the same number of cores,
...
Ignore "hyperthreaded" cores - they do NOT work well to support folding GPUs, they MIGHT be worth a QUARTER of a real core.
...
In fact, ignore AMD

AMD FX supports only pci-e v2, that, in theory, may limit gpu productivity

i3 4130 with HT enabled is enough for three 1080ti, so HT does work well to support folding GPUs

amd gpus loads CPU less than nVidia`s :
with amd gpu, FahCore uses 100% per thread on task start and and drops to ~60% per thread on main time of task count,
with nVidia, FahCore uses 100% of CPU thread whole folding time.


PCI 2.0 16x is more than enough for any GPU - even the 1080ti.
8x can lose a little bit, but commonly only a few percent at most even on the 1080ti.
Any quad core should be ENOUGH to support 4 x 1080ti - but have you priced Intel quad-cores vs the FX 6300 6-core or even the 8320e 8-core lately - and given my results as I talk about below, I'm not sure a single core is enough to support a 1080ti fully even WITH Hyperthreading enabled unless it's a very HIGH end CPU.

I tried to run *2* 1080 ti with a G4600 (Kaby Lake 2 core WITH hyperthreading, the SAME as a low-to-midrange Kaby Lake i3 except less cache) - the setup was DID CPU LIMIT on some work units, and was right on the edge most of the time on the rest.
Dedicated folding machine that wasn't running anything else, one 1080ti on a PCI-E 3.0 16x slot the other on a PCI-E 3.0 4x slot.
I can't recommend trying to run a 1080 ti on a SINGLE core - even with hyperthreading - as it gets very marginal.

AMD GPUs load the CPU a lot less in general because they're doing less work - FAH is not AMD GPU friendly, something about the differences between AMD and Nvidia GPU design make FAH work a LOT better on Nvidia GPUs.
QuintLeo
 
Posts: 39
Joined: Sat Jun 17, 2017 5:43 am

Re: Guidance for building a folding dedicated computer

Postby QuintLeo » Tue Sep 05, 2017 2:32 am

foldy wrote:The nvidia driver can only use one CPU core per GPU.
Looks like really old slow CPUs limit the new fast GPUs.

QuintLeo wrote:"My short-lived rig with a pair of GTX 1080ti running on a dual-core Intel G4600 (Kaby Lake 3.5 Ghz) WAS CPU LIMITING NOTICEABLY on both GPUs"
That is interesting, did anyone else had this problem with a new fast GPU but only dual core for 2 GPUs?

The pcie bandwidth limit is only a big problem on Windows. Using Linux you loose only some % and so using x1 risers on Linux is possible.



Machine in question was running Ubuntu 14.04 64-bit.
QuintLeo
 
Posts: 39
Joined: Sat Jun 17, 2017 5:43 am

Re: Guidance for building a folding dedicated computer

Postby Joe_H » Tue Sep 05, 2017 2:47 am

QuintLeo wrote:I tried to run *2* 1080 ti with a G4600 (Kaby Lake 2 core WITH hyperthreading, the SAME as a low-to-midrange Kaby Lake i3 except less cache) - the setup was DID CPU LIMIT on some work units, and was right on the edge most of the time on the rest.
Dedicated folding machine that wasn't running anything else, one 1080ti on a PCI-E 3.0 16x slot the other on a PCI-E 3.0 4x slot.

That is marketed as a Pentium. Limitation you did not mention is that it only supports a total of 16 PCIe lanes, so depending on what else needed PCIe lanes such as for access to storage, there would not have been enough to run one GPU at 16x and another at 4x. Quite likely your 16x GPU was actually running at 8x at best, that would depend on your board.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Joe_H
Site Admin
 
Posts: 3883
Joined: Tue Apr 21, 2009 4:41 pm
Location: W. MA

Re: Guidance for building a folding dedicated computer

Postby rwh202 » Tue Sep 05, 2017 5:58 am

Joe_H wrote:
QuintLeo wrote:I tried to run *2* 1080 ti with a G4600 (Kaby Lake 2 core WITH hyperthreading, the SAME as a low-to-midrange Kaby Lake i3 except less cache) - the setup was DID CPU LIMIT on some work units, and was right on the edge most of the time on the rest.
Dedicated folding machine that wasn't running anything else, one 1080ti on a PCI-E 3.0 16x slot the other on a PCI-E 3.0 4x slot.

That is marketed as a Pentium. Limitation you did not mention is that it only supports a total of 16 PCIe lanes, so depending on what else needed PCIe lanes such as for access to storage, there would not have been enough to run one GPU at 16x and another at 4x. Quite likely your 16x GPU was actually running at 8x at best, that would depend on your board.

Actually, the 16 lanes are just the separate dedicated graphics lanes and are typically tied to one PCI-E slot (unless you have an SLI board with switches etc.). There are further lanes provided by the chipset that provide storage and the other slots, so 16x and 4x is quite normal on pentiums. They are then connected to the CPU over the DMI.
rwh202
 
Posts: 292
Joined: Mon Nov 15, 2010 8:51 pm
Location: South Coast, UK

Re: Guidance for building a folding dedicated computer

Postby v00d00 » Tue Sep 05, 2017 4:51 pm

I have a question on PCI-E utilisation between Gen 2 and 3.

On my AMD FX i have Gen 2, my GPU PCIE utilisation is at 3-4% at any given time according to nvidia-smi, running a GTX 970.
On my Intel Skylake i7, my GPU PCIE utilisation is at 2-3% at any given time according to nvidia-smi, running on a different GTX 970.
Both systems running Slackware 14.x with a modern kernel. But I also got this same amount last year on Debian.

PPD is essentially identical between the two.

Does PCI-E gen 3 actually make any difference, because it looks like the amount of data being sent and received comes nowhere near to saturating the lanes?

As for what computer to build for dedicated, its an apples and oranges discussion. In reality it doesnt really matter. If it has a relatively modern quad core cpu, a board with at least 2 PCIE slots, a 4GB stick of ram and a decent 650-850W PSU, you are good to go for most modern graphics card from either manufacturer. I would say to go with SSD, not for the fact you need the speed, but that it uses very little power and you can generally pick up a cheap industrial grade drive (16GB is fine for OS and folding) for very little money from ebay. If your budget is unlimited you could go with 1080 Ti with either water cooling or hybrid cooling, or maybe a couple of 1060's.

Key things to consider include Power Usage, Noise and Ambient Temperature. If electricity costs a lot, then aim for something thats much more streamlined on the PPD/watt. If you live in a place where the average outside temperature is 30-40C then you may end up having to use a lot more power to keep your home cool when running folding gpu. Similarly dont run cpu client. 6000PPD on avg for an FX. My 970 uses only 60w more and does about 270k/day. Their are enough people running cpu/smp that dedicate server level systems to it. If you do it for the science and you live at the North Pole, then running cpu client is probably going to help keep your house warm. Otherwise I see it as a waste of power that could be used to run gpu.

As ever, YMMV.
Image
User avatar
v00d00
 
Posts: 508
Joined: Sun Dec 02, 2007 4:53 am
Location: In the UK

PreviousNext

Return to FAH Hardware

Who is online

Users browsing this forum: No registered users and 2 guests

cron