CPU requirement for GPU folding (NV)

A forum for discussing FAH-related hardware choices and info on actual products (not speculation).

Moderator: Site Moderators

Forum rules
Please read the forum rules before posting.
Frontiers
Posts: 50
Joined: Thu Sep 13, 2012 3:23 pm
Hardware configuration: Ryzen 5 5600x
G.Skill 2x16 GB 3200@3333
GTX 1070
Lancool II Mesh Perf.
Linux Mint 21.3
some drives, some cooler, some peripherials

Re: CPU requirement for GPU folding (NV)

Post by Frontiers »

I think that multi-GPU systems PPD degradation is because of copies of fahcore21.exe walking between CPU cores.
In rig with 4 GPUs you have 4 fahcore21.exe and all 4 doing a chinese firedrill all the way.
One of possible solutions to increase PPD in multi GPU rigs with PCIe 3.0 motherboards is Process Lasso utility: just nail 4 fahcore21.exe affinity to concrete physical CPU cores in tab of active processes in Process Lasso.
Do not nail affinity of any system or drivers processes - it could lead to very laggy Windows GUI, sometimes even near impossible to boot in and making changes.
For CPU GRO-A7 folding - one could nail it's fahcores to different physical CPU cores from those used by GPU's fahcore21.exe.
For multi-GPU systems on PCIe 2.0 boards only possible solution for PPD inproving is folding at Linux (Mint/Ubuntu or whatsoever), because with PCIe 2.0 Linux gives much higher GPU utilization with 372.20 drivers than Windows with any drivers.
But I can't give any advices how to tune multi-GPU Linux rig - because I still fold with only 1 GPU.
Aurum
Posts: 296
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: CPU requirement for GPU folding (NV)

Post by Aurum »

Hi Frontiers, Nice idea, I'll give it a try. I wonder if when a WU finishes will PL lose its lock on that CPU core???
In Science We Trust Image
Frontiers
Posts: 50
Joined: Thu Sep 13, 2012 3:23 pm
Hardware configuration: Ryzen 5 5600x
G.Skill 2x16 GB 3200@3333
GTX 1070
Lancool II Mesh Perf.
Linux Mint 21.3
some drives, some cooler, some peripherials

Re: CPU requirement for GPU folding (NV)

Post by Frontiers »

Duce H_K_ wrote:it allows GTX1070 @ 2.1GHz with p9431 folding load up to 82% while up to 85% without any heavy CPU occupation
You need to try Linux Mint with that how-to from tguskill (great thanks to him) from overclockers.ru board.
I saw same low GPU utilization under Windows with PCIe 2.0 and about 96-97% GPU utilization under Linux Mint 18.1 with same PCIe 2.0 motherboard.
This higher GPU utilization gives 200k more PPD under Linux Mint with same GTX 1070, clocked 1950 MHz in Linux instead of 2050 MHz in Windows 10.
Beacuse of so much higher GPU utilization - there is no need for such high overclocking under Linux as it was under Windows, as rule of thumb - overclock it 100 MHz lower than it was under Windows, with unlocking Power Loimit with "sudo nvidia-smi -i0 -plimit XX0" from console, XX0 - design power limit of your board.
Aurum wrote:I wonder if when a WU finishes will PL lose its lock on that CPU core???
If you change process affinity with Windows Task Manager - folding process will lose affinity after fahcore restarted with new WU.
If you change process affinity with Process Lasso - you will have an option (second or third from the top at right click menu) to make constant affinity - it will not change after fahcore restarted with new WU.
Aurum
Posts: 296
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: CPU requirement for GPU folding (NV)

Post by Aurum »

Ok I installed on a 4 GPU headless rig. E5-4640v3 with 12c24t. At first I tried only using 4 CPU threads for the GPUs but they were constantly pegged at 100%.
Then I set the last 8 CPUs to FahCore_21.exe and the first 16 CPUs to FahCore_a4.exe. Manually set FAHControl to use 16 CPUs. This gives total CPU usage ~87% per Win7 Task Manager. The 4 GPUs don't need all 8 threads so that leaves headroom for other stuff to run without having to leave an unassigned CPU thread. WUs were already running so it's too early to tell but the estimated PPD has increased 2%. In an hour everyone will have a new WU and the test can begin.
In Science We Trust Image
Duce H_K_
Posts: 113
Joined: Mon Nov 09, 2015 3:52 pm
Hardware configuration: MoBo•Gigabye X99 UD4-CF F24
CPU•<UPD 20.05.2023>Xeon V3 2680 V4 14c28t 35Mb L3
RAM•DDR4 Hynix 2133 CL14 4*16 DualRank Quad channel
HDD•ST1000DM003 Sata3 NCQ
GFX•GT220
PSU•Chieftec GPS750C 80+ Gold after repair
Cooling•Air 2xDeepCool UF120

Internet•200Mbit/s FTTB↓ white dynamic, ERTH, router RB951G-2HnD

Other•Redmi 7A <runs WUProp :-/>
Location: Russia
Contact:

Re: CPU requirement for GPU folding (NV)

Post by Duce H_K_ »

Till Nov, 27 I expect second GPU, the Palit SJS 1080Ti which will be available for me. So I meant single GPU configuration. No heat related issues for last few months.
12c24t means HT=on of course 'cause CPU tasks are not doing really great without it. CPU model is 2658 as a seller wrote but I cannot read it directly from hardware reports (CPU-z).
My former hardware was AMD A8 5600k CPU @ Asus A-55M MoBo. Didn't get much luck with Mint installation inside VMWare from tguskill's manual. Despite my hard efforts GPU couldn't be seen by virtual machine. Cannot fully refuse Windows. Planning to try once again with current newer H/W, ETA of 2 months or so. I want to thank everybody for the recommendation.
To assign threads to 21 & a* FahCores more or less properly I use WinAFC with sub parameter assign=2 or assign=3

Code: Select all

*\FahCore_21.exe := CPU0+CPU1+CPU2 [priority=High, ssign=2]
*\FahCore_a*.exe := CPU1+CPU2+CPU3+CPU4+CPU5+CPU6+CPU7+CPU8+CPU9+CPU10+CPU11+CPU12+CPU13+CPU14+CPU15+CPU16+CPU17+CPU18+CPU19+CPU20+CPU21+CPU22+CPU23 [priority=BelowNormal]
But AFAIK this sample is not as precisely as it is needed.
EDIT CPU mode 4c8t @ 2.6GHz provided 780kPPD with p9414, p9415 and 730kPPD with p9430 project. That are the maximums.
   510 290 819 pts earned in Folding@home project
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: CPU requirement for GPU folding (NV)

Post by bruce »

Assigning 1-23 to _a* means that only cpu0 is "free" for _21. That also suggests that NOTHING ever interrupts it's use of cpu0, which is probably not true. Giving _21 three choices may seem like a good idea, but that also suggests that you actually expect there to be times when it might use cpu1 or cpu2. Any such sharing of resources will degrade your performance rather than increase it.

I don't see where you've posted FAH's config so I have to assume that you've configured _a* into multiple slots so that prime factors are avoided -- such as CPU:16 + CPU:6. There, too, you'll reduce performance if you try to use VM features to enable CPUs to be shared resources between more than one WU.
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: CPU requirement for GPU folding (NV)

Post by foldy »

I made an experiment with 6 core/12 thread CPU and gtx 1080ti on Windows 10.
1) I could assign 10 threads to CPU work and leave 1 thread for GPU and 1 thread for OS without sacrificing GPU speed.
2) When using 11 threads for CPU work and 1 thread for GPU then GPU speed goes down.
3) When using 11 threads for CPU work and 1 thread for GPU and set FahCore_21 priority to high then GPU speed is only slightly reduced.

I guess for many core/thread CPU the best option to get best GPU performance is to set the FahCore_21 to high priority. Then it doesn't matter if all other threads are used at normal priority or below by CPU folding or something else. Maybe this could become an option in FahClient to set GPU slots FahCore_21 CPU priority to high? (There is currently such option for CPU core to increase priority from "low" to "lower than normal" but that does not help in this scenario)

Currently FahClient makes CPU usage thread count -1 for each GPU. Maybe that should be true only if core count = thread count and else make another -1 for OS?
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: CPU requirement for GPU folding (NV)

Post by bruce »

foldy wrote:Currently FahClient makes CPU usage thread count -1 for each GPU. Maybe that should be true only if core count = thread count and else make another -1 for OS?
The default setting is established ONLY at the time FAH is installed. FAH's code is not designed to override your choices if you choose to make poor choices. V7.4.4 will let you set it to 11 if you wish to, but that's a bad choice, not because it considers leaving another -1 for the OS, but because it's a prime number which will produce a high failure rate. V7.4.15+ will reduce it further if necessary to obtain a setting that is more likely to be assigned work and less likely to fail.
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: CPU requirement for GPU folding (NV)

Post by foldy »

OK I see the prime number problem and the FahClient V7.4.15+ beta was adapted to not use prime numbers.

But for a 8 core/16 thread CPU and one GPU the default after FAH installed will be 16-1 = 15 FAH CPU cores usage. Not a prime but no thread for OS left and no high priority for GPU core's CPU part.

So my example is to illustrate maybe the FahClient default installation needs another adaptation to optimize GPU usage in general?
Aurum
Posts: 296
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: CPU requirement for GPU folding (NV)

Post by Aurum »

foldy wrote:Maybe this could become an option in FahClient to set GPU slots FahCore_21 CPU priority to high? (There is currently such option for CPU core to increase priority from "low" to "lower than normal" but that does not help in this scenario)
That was one of the suggestions we'd made for F@H many moons ago. Maybe some day.
In Science We Trust Image
FldngForGrandparents
Posts: 70
Joined: Sun Feb 28, 2016 10:06 pm

Re: CPU requirement for GPU folding (NV)

Post by FldngForGrandparents »

One other thing to note. If it is a dedicated machine with multi GPU it is best to turn off Hyperthreading. Otherwise fahcore can jump to using the same physical core on both threads. Hyperthreading is only good for apps that do a lot of IO and are threaded.
Image

Dedicated to my grandparents who have passed away from Alzheimer's

Dedicated folding rig on Linux Mint 19.1:
2 - GTX 980 OC +200
1 - GTX 980 Ti OC +20
4 - GTX 1070 FE OC +200
3 - GTX 1080 OC +140
1 - GTX 1080Ti OC +120
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: CPU requirement for GPU folding (NV)

Post by bruce »

It's your choice whether to turn off HT.

Based on rather old measurements (probably with a core2duo) using both halves of a single CPU will accomplish about 15% more than using just one half. You're correct that Apps that do a lot if I/O benefit most from HT but FAH's performance is based ~85% on the shared floating point (/SSE/AVX/etc.) registers, not 100%.
Aurum
Posts: 296
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: CPU requirement for GPU folding (NV)

Post by Aurum »

FldngForGrandparents wrote:...a dedicated machine with multi GPU it is best to turn off Hyperthreading. Otherwise fahcore can jump to using the same physical core on both threads. Hyperthreading is only good for apps that do a lot of IO and are threaded.
I'm trying this using Project Lasso. I let 4 GPUs have the last 8 threads (20-27) & set FahCore_21 priority to High. The Xeon has 14c28t so I let a4 etc have threads 0-19. I configured CPU to use 20 threads in FAHControl. CPUs 0-19 are always pegged so I may have to drop to 18 CPUs. Initially PPD dropped when making changes in mid-WUs.
In Science We Trust Image
Aurum
Posts: 296
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: CPU requirement for GPU folding (NV)

Post by Aurum »

Built a new rig with an E5-2699v4 with 22c44t but F@H can only recognize 32 threads max. If I try to set it higher I get this error message:

Code: Select all

On client "Rig-06" 192.168.1.206:36330: Invalid value for option 'cpus'.  Option help:
  --cpus <integer=-1>
      How many CPUs a slot should use. <= 0 will use all the CPUs detected in the system.
Caused by: 38 is greater than maximum value 32
Hmm, Task Manager shows all 44 threads working on something. I'll try disabling hyperthreading in the BIOS and see what happens.
In Science We Trust Image
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: CPU requirement for GPU folding (NV)

Post by foldy »

It can make a difference if you give last 8 threads or first to the GPUs because the OS by default also gives the first or last threads to any tasks.
Post Reply