Hello everyone!
I've been folding for a few months now on an extra system I had. The system is comprised of an i5 2500k @4.3GHz stock voltage, GTX 970 from MSI and 8Gb of RAM. Plus I have a GTX 960 2Gb that was around so I stuck that in there but Windows 10 being windows 10, it didn't want to work.
Therefore I opened up a Dell slim unit with an E8300 CPU, added a 430w PSU that I shorted to power the GTX 960 and ran both systems with the same name, passkey, etc.
I was getting 300k on the 970 and 170k on the 960.
The other day I got around to DDU-ing (Display Driver uninstall) Windows 10 in the i5 machine, installed both cards, got the latest drivers and it works, I can now fold on both cards at the same time. Thing is, my total PPD is only 440k. Not the 470k I was expecting.
The i5 2500k is now running at a constant 50%ish usage, and I wanted to know if that was the bottleneck. FYI I'm not folding on the CPU, I know it's kinda useless.
The next revision of this system will happen in 2 to 3 months when I get back another system I borrowed to a friend with a Xeon 2660 8c/16t with 24Gb of RAM and the GPUs will be 2xGTX 970 to fold. If the i5 2500k is the bottleneck, will the Xeon 2660 bottleneck the system too?
(Note: if my PSU can hold it up, I'll add the GTX 960 to fold in the same CP, so 2x970 and 1x960 in this system, no CPU folding).
It's possible I didn't make myself clear, if so, let me know!
Best regards and looking forward to your advised answers,
seecker
Team 229844 (PC in question is SeeckerPhiPhi)
Multiple different GPUs and CPU bottleneck
Moderators: Site Moderators, FAHC Science Team
-
foldy
- Posts: 2061
- Joined: Sat Dec 01, 2012 3:43 pm
- Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441
Re: Multiple different GPUs and CPU bottleneck
i5 2500k is not the bottleneck as it has 4 cores and you run 2 GPUs without CPU slot and each GPU needs 1 CPU core to feed it. Xeon 2660 8 core is perfect too.
As you use Windows pcie could be limit a little as i5 2500k only can do pcie 2.0 with 16 lanes and with dual GPU you run them with pcie gen2 x8 = pcie 3.0 x4 or less. But for the slower GPUs 970 and 960 that should be still enough. But the 970 in the faster pcie slot and 960 in the slower pcie slot if their speed is different. Xeon 2660 will be pcie gen3 with 40 lanes, so this could even drive 4 highend GPUs at pcie gen3 x8 without any loss on Windows.
(On Linux pcie speed doesn't matter much and can even run GPUs on fast x1 risers)
Another factor is heat generated by 2 GPUs is more challenging for cooling, especially if you installed one GPU above the other in a closed case. So maybe the top GPU is downclocking a little and needs some case fans to help out?
But the real answer I guess is there is no hardware bottleneck but the 7% PPD loss you see maybe just is work unit or projects are little different.
I would not worry.
As you use Windows pcie could be limit a little as i5 2500k only can do pcie 2.0 with 16 lanes and with dual GPU you run them with pcie gen2 x8 = pcie 3.0 x4 or less. But for the slower GPUs 970 and 960 that should be still enough. But the 970 in the faster pcie slot and 960 in the slower pcie slot if their speed is different. Xeon 2660 will be pcie gen3 with 40 lanes, so this could even drive 4 highend GPUs at pcie gen3 x8 without any loss on Windows.
(On Linux pcie speed doesn't matter much and can even run GPUs on fast x1 risers)
Another factor is heat generated by 2 GPUs is more challenging for cooling, especially if you installed one GPU above the other in a closed case. So maybe the top GPU is downclocking a little and needs some case fans to help out?
But the real answer I guess is there is no hardware bottleneck but the 7% PPD loss you see maybe just is work unit or projects are little different.
I would not worry.
Re: Multiple different GPUs and CPU bottleneck
Hello foldy, and thanks for your answer!
I don't know how to run Linux, that's why I stick with Windows.
The top card is the GTX 970 and it isn't throttling at all. it's running at 75°C for 72% fan speed. 1328 on the core and 3005 on the memory.
The 960 on the other hand is running 67°C for 40% fan speed. 1353 on the core and 3005 on the memory.
Both power sliders are at max.
The case i use isn't well ventilated, but I removed the side panel anyway.
Waiting to get the Xeon system
I don't know how to run Linux, that's why I stick with Windows.
The top card is the GTX 970 and it isn't throttling at all. it's running at 75°C for 72% fan speed. 1328 on the core and 3005 on the memory.
The 960 on the other hand is running 67°C for 40% fan speed. 1353 on the core and 3005 on the memory.
Both power sliders are at max.
The case i use isn't well ventilated, but I removed the side panel anyway.
Waiting to get the Xeon system
Re: Multiple different GPUs and CPU bottleneck
I don't think the corei5 is much of a bottleneck.
Even with hyperthreading, they should take 3 graphics cards easily.
The bottleneck is probably the PCIE speed.
Before upgrading to a Xeon system, perhaps it is better to upgrade the graphics card instead.
As a good Xeon motherboard goes in the several hundreds of dollars.
A xeon, in most cases, is overkill for folding.
They have 32 PCIE lanes, but no guarantee your motherboard uses all of them.
If you can find a motherboard that offers 8 PCIE slots at 4x speed, you could run a massive folding machine (think one that consumes 1500W on the wall, and gets you 10+M PPD).
Most of the time, you'll find Xeon motherboards offer only 2 or 3 full sized slots, sometimes only 1, with 2 or 3 PCIE 1x slots.
In such cases it's better to just run a regular Core i3, i5, or i7, as they're cheaper.
I myself am looking to downgrade my folding system.
Upgrade the graphics card, cause that's where all the power comes from.
Run 2x RTX 2080 cards ($500 a piece) with a $50 Celeron G4900 CPU, on a $50 motherboard, that offers 2 slots.
And that cheap system with 2x expensive graphics cards, should get me ~3M PPD for about 500W energy consumption.
Take it from a guy who is currently running a Xeon system.
It's not the CPU, it's the graphics card that can make the system.
If I were you, I'd look into swapping the slowest graphics card with an RTX 2060, 2070, or 2080.
or a 1660 ti worst case.
It'll net you more PPD than swapping out your CPU to a Xeon.
Even with hyperthreading, they should take 3 graphics cards easily.
The bottleneck is probably the PCIE speed.
Before upgrading to a Xeon system, perhaps it is better to upgrade the graphics card instead.
As a good Xeon motherboard goes in the several hundreds of dollars.
A xeon, in most cases, is overkill for folding.
They have 32 PCIE lanes, but no guarantee your motherboard uses all of them.
If you can find a motherboard that offers 8 PCIE slots at 4x speed, you could run a massive folding machine (think one that consumes 1500W on the wall, and gets you 10+M PPD).
Most of the time, you'll find Xeon motherboards offer only 2 or 3 full sized slots, sometimes only 1, with 2 or 3 PCIE 1x slots.
In such cases it's better to just run a regular Core i3, i5, or i7, as they're cheaper.
I myself am looking to downgrade my folding system.
Upgrade the graphics card, cause that's where all the power comes from.
Run 2x RTX 2080 cards ($500 a piece) with a $50 Celeron G4900 CPU, on a $50 motherboard, that offers 2 slots.
And that cheap system with 2x expensive graphics cards, should get me ~3M PPD for about 500W energy consumption.
Take it from a guy who is currently running a Xeon system.
It's not the CPU, it's the graphics card that can make the system.
If I were you, I'd look into swapping the slowest graphics card with an RTX 2060, 2070, or 2080.
or a 1660 ti worst case.
It'll net you more PPD than swapping out your CPU to a Xeon.
Re: Multiple different GPUs and CPU bottleneck
Thanks for the input MeeLee!
I know a Xeon is kinda overkill for folding, but I already have the system, I've bought it a year ago for testing. The motherboard I'm going to use is the Asus P9x79 WS, it has more full size PCI-e slots that I can shake a stick at ^^ (6 PCI-e 16X slots! Quad 8x and 2 4x!).
My goal isn't to fold al much as possible. Right now I don't want to put money in the project at all. I didn't buy a new case or anything except for a 1500W PSU that cost me 100€ (Yes € not $, I'm French). I'm just having fun and killing time with an overkill system. I'm not going to change the GPUs for better cards. If I have to buy a GPU it would be to upgrade my GTX 1060 6Gb in my main PC (not used to fold) but I don't feel the need to change my config.
I won't fold on the Xeon itself as well.
I don't think I'll be folding with it for years either. There is a chance that I'll have to move in Septembre, so I might sell the system because my girlfriend won't accept having a PC run 24/7 anywhere in the appartment.
For the story, I currently have 3 systems that I don't really use right now: the Xeon with an AsRock mobo with a GTX 690, an i7 3820 build with the P9x79 and a GTX 970 that another friend uses for gaming and then the i5 2500 with GTX 970 and 960 (plus the modified Dell OEM with the GTX 760 attached to it).
In a few months I'll change all the configs around: I'm keeping the Xeon and the P9x79, putting the 2 GTX 970 and the GTX 670 (and a GTX 660 too perhaps because why not), my first friend will keep the Asrock mobo but get the i7 3820 (he bought a GTX 970 to replace the GTX 670), The GTX 760 goes to my girlfriend to replace her GTX 660, and my second friend will get the i5 2500k with the GTX 960.
It SEEMS complicated but it's just a bit carrousel of parts and builds I have and don't really want to sell because my friends needed computers and didn't have the money to buy good hardware (hell, took my friend 6 months of saving to gather the 100e for the GTX 970).
Oh and before talking about power consumption: I have a fixed amount on my rent for water and electricity, so whatever I consume, it stays the same. So I don't care AT ALL about being efficient ^^ The only problem is cooling my 1 room studio because the window doesn't provide enough airflow sometimes...
Yeah, first world problems lol.
Just for the fun of it I did install the folding client on my PC at work that has a Quadro P4000. This thing kicks ass to me. The GPU alone gives about 500k PPD!
PS: I know that CPU folding isn't worth it, but what PPD do you think I could get (limiting the threads if needed to keep the GPUs fed). It's just an 8c/16t CPU at 3GHz that makes about 1050cb ish in Cinebench R15.
I know a Xeon is kinda overkill for folding, but I already have the system, I've bought it a year ago for testing. The motherboard I'm going to use is the Asus P9x79 WS, it has more full size PCI-e slots that I can shake a stick at ^^ (6 PCI-e 16X slots! Quad 8x and 2 4x!).
My goal isn't to fold al much as possible. Right now I don't want to put money in the project at all. I didn't buy a new case or anything except for a 1500W PSU that cost me 100€ (Yes € not $, I'm French). I'm just having fun and killing time with an overkill system. I'm not going to change the GPUs for better cards. If I have to buy a GPU it would be to upgrade my GTX 1060 6Gb in my main PC (not used to fold) but I don't feel the need to change my config.
I won't fold on the Xeon itself as well.
I don't think I'll be folding with it for years either. There is a chance that I'll have to move in Septembre, so I might sell the system because my girlfriend won't accept having a PC run 24/7 anywhere in the appartment.
For the story, I currently have 3 systems that I don't really use right now: the Xeon with an AsRock mobo with a GTX 690, an i7 3820 build with the P9x79 and a GTX 970 that another friend uses for gaming and then the i5 2500 with GTX 970 and 960 (plus the modified Dell OEM with the GTX 760 attached to it).
In a few months I'll change all the configs around: I'm keeping the Xeon and the P9x79, putting the 2 GTX 970 and the GTX 670 (and a GTX 660 too perhaps because why not), my first friend will keep the Asrock mobo but get the i7 3820 (he bought a GTX 970 to replace the GTX 670), The GTX 760 goes to my girlfriend to replace her GTX 660, and my second friend will get the i5 2500k with the GTX 960.
It SEEMS complicated but it's just a bit carrousel of parts and builds I have and don't really want to sell because my friends needed computers and didn't have the money to buy good hardware (hell, took my friend 6 months of saving to gather the 100e for the GTX 970).
Oh and before talking about power consumption: I have a fixed amount on my rent for water and electricity, so whatever I consume, it stays the same. So I don't care AT ALL about being efficient ^^ The only problem is cooling my 1 room studio because the window doesn't provide enough airflow sometimes...
Yeah, first world problems lol.
Just for the fun of it I did install the folding client on my PC at work that has a Quadro P4000. This thing kicks ass to me. The GPU alone gives about 500k PPD!
PS: I know that CPU folding isn't worth it, but what PPD do you think I could get (limiting the threads if needed to keep the GPUs fed). It's just an 8c/16t CPU at 3GHz that makes about 1050cb ish in Cinebench R15.
-
foldy
- Posts: 2061
- Joined: Sat Dec 01, 2012 3:43 pm
- Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441
Re: Multiple different GPUs and CPU bottleneck
8 core CPU with AVX support i guess 100k PPD
Re: Multiple different GPUs and CPU bottleneck
Wow that much? I didn't expect that. It's more then the GTX 760 offers (around 80k PPD).foldy wrote:8 core CPU with AVX support i guess 100k PPD
Again, out of curiosity, are there people that use dual socket motherboards to fold on Xeons with 10 cores and such and use the PCI-e ports for GPUs all in 1 system?
Come to think of it, the Xeon W-2123 I have at work produces around 37k PPD is seems...
-
Joe_H
- Site Admin
- Posts: 7867
- Joined: Tue Apr 21, 2009 4:41 pm
- Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2 - Location: W. MA
Re: Multiple different GPUs and CPU bottleneck
You mention that it is an 8-core Xeon 2660, that would make it a Sandy Bridge based processor from 2012. My estimate would be a bit more conservative, on a Core A7 WU about 70-75k PPD.
If you are running just CPU folding on it, then you could assign all 16 threads to CPU folding. But if some cores are going to be needed to support GPU folding the maximum you could use is probably 12. You would have to check to see if that provided better throughput for CPU folding than just assigning the 8 main cores to CPU folding and leaving the HT cores to support everything else.
If you are running just CPU folding on it, then you could assign all 16 threads to CPU folding. But if some cores are going to be needed to support GPU folding the maximum you could use is probably 12. You would have to check to see if that provided better throughput for CPU folding than just assigning the 8 main cores to CPU folding and leaving the HT cores to support everything else.
iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Re: Multiple different GPUs and CPU bottleneck
I'm not very experienced in the optimization departement. I don't even know if it's worth it to fold on the CPU. If I go full on 'insane' on this thing, it'll look like a mining rid at the end of the day. PCI-e risers, GPUs and big PSU(s). Since the motherboard has 6 PCI-e slots, I might use all of them in a custom case optimised for airflow. It would be fun to make.Joe_H wrote:If you are running just CPU folding on it, then you could assign all 16 threads to CPU folding. But if some cores are going to be needed to support GPU folding the maximum you could use is probably 12. You would have to check to see if that provided better throughput for CPU folding than just assigning the 8 main cores to CPU folding and leaving the HT cores to support everything else.
If I start folding on the Xeon (on all cores and threads, simple auto detect) while the Xeon as to manage the GTX 970 x2 and the GTX 670 and so on, will there be a reduction in PPD for the GPUs?
-
foldy
- Posts: 2061
- Joined: Sat Dec 01, 2012 3:43 pm
- Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441
Re: Multiple different GPUs and CPU bottleneck
Yes each GPU needs one CPU thread to feed it and FAH automatically reduces CPU slot threads by GPU count.
Re: Multiple different GPUs and CPU bottleneck
Oh ok thanks!foldy wrote:Yes each GPU needs one CPU thread to feed it and FAH automatically reduces CPU slot threads by GPU count.