Folding Forum

It really may be the GPU clock speed going down because of heat.
e.g. base clock of gtx 1080 may be 1500 Mhz but overclocked boost clock could be 2000 Mhz.
So this is a 30% difference.

ComputerGenie wrote:

Aurum wrote:...
The downside of this MB is that they placed the M.2 SSD socket under GPU1 and it chronically overheats. I've got a 120 mm fan sitting on top of the cards blowing down but it's not enough. I have a wall of three 120 mm fans across the back of the cards as well. I'm going to try adding a fan on the side.

https://www.amazon.com/gp/product/B01E8YS122 << I have an "open air" rig, but I use 1 of those (plugged into MB and blowing "front to back") to cool 2 1080s in a room that sometimes hits above 90°F.

I've got some of those old AntMiner fans collecting dust, they sure were noisy. I'm thinking of something like this, a thermoelectric cooler
https://www.amazon.com/TEC1-12706-Therm ... ric+cooler
I cut the slats out of the mounting bracket between cards and put a second 120 mm fan on top. Problem seems to be getting air flow into the blackbody cavity where the M.2 SSD sits. So many fans may just be creating turbulence. I'm thinking maybe a fish tank air pump with the Tygon tube pointing at the SSD.
Maybe I'll wrap the M.2SSD with 12 gauge copper wire and then wrap the Cu wire around the cooling pipes of the EVO 212

foldy wrote:It really may be the GPU clock speed going down because of heat.
e.g. base clock of gtx 1080 may be 1500 Mhz but overclocked boost clock could be 2000 Mhz.
So this is a 30% difference.

I think you're right but it's hard to monitor with GPU-Z reporting a different GPU frequency than Gigabyte Xtreme Gaming control shows, I'm going with the latter since I can see a change quickly when I apply a change.

EVGA Precision XOC is really nice because it shows all cards on a single GPU frequency graph with color-coded lines. Gigabyte shows a single screen at a time with only one card displayed.

Right now the overheating problem is so bad it turns off the rig. That's what you get with a $124 X99 mb.

Mega folder PS3EdOlkkola recommended to build only 3 GPU PC with X99 mainboards having extra space between the pcie slots
viewtopic.php?f=38&t=28847&start=105#p292607
I don't know if you can switch your mainboard?

Did you try to set all fans to 100% or is noise a problem for you?

Are you using the M2 SSD via PCIE although all 40 lanes are used by GPUs?
Sorry, if I don't understand properly

A question that I may have missed the answer to: Is this a box rig or "open air"?

foldy wrote:Mega folder PS3EdOlkkola recommended to build only 3 GPU PC with X99 mainboards having extra space between the pcie slots
viewtopic.php?f=38&t=28847&start=105#p292607
I don't know if you can switch your mainboard?

Did you try to set all fans to 100% or is noise a problem for you?

Mega Ed can afford to splurge, he also uses Titan X Pascal cards. I was trying to get as much PPD/rig$ as possible.

foldinghomealone wrote:Are you using the M2 SSD via PCIE although all 40 lanes are used by GPUs?
Sorry, if I don't understand properly

The photo to right of the following description looks just like my rig but I also have a Coolmaster Hyper T4 on the CPU.
http://uk.gigabyte.com/Motherboard/GA-X99-SLI-rev-10#ov
"3-Way / 4-Way Graphics with Premium PCIe x16 Lane Up to *320Gb/s Bandwidth
GIGABYTE X99 motherboards feature a unique PCI Express design that utilizes 100% of all 40 lanes from the CPU when in a 4-Way or 3-Way graphics configuration. Standard designs limit all 4 main PCIe lanes to x8 (64Gb/s) bandwidth, but thanks to the onboard external clock generator combined with the direct connection of one of the x16 lanes to the CPU (switchless design), GIGABYTE X99 motherboards can unlock the full bandwidth available and provide the best graphics bandwidth possible to the user."
The manual has a block diagram explaining it in more detail.

When I said "turbulence" it gave me an idea. I had the T4 fan pushing on the cooling fins but sitting in the front (front to me is the business end where you plug in cables since I do NOT use cases). I have three 120 mm fans on back of the tray blowing forward so it just made a turbulent mess back there and the card (GPU0) sitting next to the CPU was the one overheating. I just reversed the T4 fan to be a pulling fan going in the same direction as the rear fans. The temperature of the M.2SSD (still at 61C) dropped 10C
The CPU & MB temps dropped as well as GPU0 (which is still running 82C).

ComputerGenie wrote:A question that I may have missed the answer to: Is this a box rig or "open air"?

I use old Lian Li MB trays to build my open-air headless multi-GPU rigs. This photo shows one mounted on top of Lian Li test bed:
https://images.bjorn3d.com/Material/rev ... /pic12.jpg

Aurum wrote:I use old Lian Li MB trays to build my open-air headless multi-GPU rigs. This photo shows one mounted on top of Lian Li test bed:

Then are you sure that the fan on "top of the cards blowing down" isn't adding to the issue (all of my 1080s blow exhaust up sitting in that position).

ComputerGenie wrote:

Aurum wrote:I use old Lian Li MB trays to build my open-air headless multi-GPU rigs. This photo shows one mounted on top of Lian Li test bed:

Then are you sure that the fan on "top of the cards blowing down" isn't adding to the issue (all of my 1080s blow exhaust up sitting in that position).

I have to agree, after much experimentation I found that sitting a 120mm fan on top of the gpu's sucking air up from between the cards and exhausting into the open was much better.

As you have 4 GPUs in a row maybe a very big 230mm fan could cover all?
http://www.xoxide.com/250mm-case-fans.html
e.g. http://www.xoxide.com/bitfenix-spectrep ... black.html
(Did not try this myself)

I have 1xXeon E5 V3 ES 12c24t @ 2.4GHz busy with GRO_A7, it allows GTX1070 @ 2.1GHz with p9431 folding load up to 82% while up to 85% without any heavy CPU occupation
Barely saw 94** WUs giving 710-752kPPD in 4c8t @ 2.6 GHz variant. P.S. Gigabyte UD4 MoBo, PCI-E 3.0 x16 no other cards are plugged including M2

Duce H_K_ wrote:I have 1xXeon E5 V3 ES 12c24t @ 2.4GHz busy with GRO_A7, it allows GTX1070 @ 2.1GHz with p9431 folding load up to 82% while up to 85% without any heavy CPU occupation
Barely saw 94** WUs giving 710-752kPPD in 4c8t @ 2.6 GHz variant. P.S. Gigabyte UD4 MoBo, PCI-E 3.0 x16 no other cards are plugged including M2

Hi, Trying to understand what you're asking. Which E5-xxxxV3 do you have??? So you have one 1070 on your UD4. Looks like it's getting good PPD. I configure my CPUs to use n-2 threads for CPU folding and that gives one thread for the GPU and another for MB use. If I look at Task Manager Performance I like to see some head room. On this MB you can use an M2 and it only disables a couple of SATA plugs and does not slow down the GPU PCIe bus. I can run 4 EVGA 1080 Ti on my GA-X99-UD4 just fine.

Duce H_K_ wrote:I have 1xXeon E5 V3 ES 12c24t @ 2.4GHz busy with GRO_A7...

Are we to assume you're talking about Windows or Linux? Are we to assume HyperThreading is enabled?

By default, if you have two GPUs, FAHCore_a4 / _a7 should configure itself to use no more than N-2 CPUs (in your case, that would be 22 out of 24 CPUs) -- and in many cases it will allocate less. FAHCore_21 is expected to use the other two CPUs, one for each GPU. This assumes that nothing else is using a measurable amount of CPU resources on a continuous basis. Leaving some additional headroom is a good idea.

Degradation will occur if, when if you pause both GPUs (stopping 0x21), the Windows Task manager reports anything exceeding 91.6% = 22/24. It doesn't matter what is using the CPUs. If your system has a regular CPU load that keeps more CPUs busy than that, you'll get better GPU performance if you reduce the load either by suspending non-FAH processing or reducing the number of CPUs allocated to FAHCore_a7 / _a4