Multi-GPU

ITfanboy · Post by **ITfanboy** » Fri Apr 03, 2020 2:14 pm

Is there a way to focus Multi-GPU on one project at a time and not each GPU get assigned a separate project? ( I'm using something similar to a mining rig with a RX 580, RX 570, 2x RX 470 and a RX 560 )

Post by **Joe_H** » Fri Apr 03, 2020 2:32 pm

No, unfortunately. The communication between the cards is not fast enough, basically each GPU is considered a separate device to assign work to. They may at times get different WUs from the same project, or at other times different projects. But the WUs are independent of each other and will have different Run, Clone and Generation numbers.

ITfanboy · Post by **ITfanboy** » Fri Apr 03, 2020 3:49 pm

Joe_H wrote:No, unfortunately. The communication between the cards is not fast enough, basically each GPU is considered a separate device to assign work to. They may at times get different WUs from the same project, or at other times different projects. But the WUs are independent of each other and will have different Run, Clone and Generation numbers.

I was hoping I could get more running and get more done but Guess I'm going to be hit and miss due to network and system limitations. hopefully they can get this program a little smother and the network better. Most the time I have at least 3 CPU or GPU units with no work. After COVID is over I will probably lower the clocks and power down to fit my budget but keep the system up during the night in the summer and 24/7 in the winter.
Thank you.

iceman1992 · Post by **iceman1992** » Fri Apr 03, 2020 4:32 pm

When the work unit shortage problem is resolved you shouldn't have a problem keeping all of them busy.
This has only happened recently because of the sudden influx of new donors.
If all GPUs have a work unit then it doesn't make a difference whether they're working on one thing or working on separate things (in terms of contribution).

Anyway I'm glad to hear you'll stay after COVID is over, welcome to F@H!

ITfanboy · Post by **ITfanboy** » Fri Apr 03, 2020 9:12 pm

iceman1992 wrote:When the work unit shortage problem is resolved you shouldn't have a problem keeping all of them busy.
This has only happened recently because of the sudden influx of new donors.
If all GPUs have a work unit then it doesn't make a difference whether they're working on one thing or working on separate things (in terms of contribution).

Anyway I'm glad to hear you'll stay after COVID is over, welcome to F@H!

Been meaning to get this done for some time and yes it's not so much I want them all running on one project. It's more of me wanting the most work without me having cards idle and me needing to que the network more often. Cards sitting idle are not using a lot of power but each one that is idle is wasting 100% of that power.
Think you for you time

Post by **PantherX** » Fri Apr 03, 2020 9:23 pm

ITfanboy wrote:...It's more of me wanting the most work without me having cards idle and me needing to que the network more often. Cards sitting idle are not using a lot of power but each one that is idle is wasting 100% of that power...

Welcome to the F@H Forum ITfanboy,

I am aware that the F@H team is working hard to increase their server capacity so this is a temporary issue. When your GPUs aren't folding (idle), they will automatically down-clock and won't be using much power. The best approach is just leave it running in the background and GPUs will automatically be assigned work the second they can

ITfanboy · Post by **ITfanboy** » Sat Apr 04, 2020 2:08 am

As of Saturday I will have 1 Vega 64, with a Threadripper 1920X, on one system on the other system RX 570, 2x RX 470, RX 580 and a RX 560 on 2x Xeon E5 2609 all low powered bios with a modded PPT on each card. even Idle for a month is going to probably increase my power over $100. May just wait until they get it fixed before i throw that much power into it as this is 2 out of 3 of my systems now one the RDT can't connect for some reason
photo at link
https://onedrive.live.com/?cid=A8CF2D05 ... 12&o=OneUp

Post by **PantherX** » Sat Apr 04, 2020 2:33 am

In that case, my suggestion would be to only have 1 rig which is generates the most PPD left turned on and you can turn off the rest for about a week or so until the F@H Servers are able to meet the demand in a reliable manner.

ITfanboy · Post by **ITfanboy** » Sun Apr 05, 2020 12:08 am

Got it set up today with just the 2 cards in and 1 CPU disabled but loaded all the drivers and numbered the other cards to the proper slots so as soon as the network is ready it's only going to take 5 minuets to get the system back up. Also changed the airflow where it pulls in air from the A/C but have a shroud and vented the exhaust with 2x 200mm fans and 8 Inch A/C out the building also has a flap so id for some reason there is a power outage it will close and reduce the chance of moisture. It's a lot cooler and saving power. A/C was running all the time Has turned on 2 times for 10-15 minuets in about 3 hours. Thanks for all of your help . My Brain is an old but good machine you where the grease that my gears to turning.

Nuitari · Post by **Nuitari** » Mon Apr 20, 2020 4:07 pm

How are you getting the multiple card setup working?

I'm looking at converting both of the smallish rigs to FAH full time, but the core seems to be unable to start if the gpu is connected to a 4 pcie to 1 pcie card (like this https://www.amazon.ca/Ubit-Extender-Bit ... B07DWWZKH4 )

I've had success with the simpler riser though.
The other rig is hitting the ram limit, so I'm waiting for some extra ram to come in.

In total there is 2x rx560, 1x rx560 OC, 8x rx570. + the CPUs (A8-9600 APU (also has a smallish gpu), and AMD X4 925)

Post by **PantherX** » Tue Apr 21, 2020 3:21 am

Nuitari wrote:How are you getting the multiple card setup working?..

The most important is budget/hardware IMO. Motherboard with support for multiple PCIe slots and sufficient PCIe lanes along with individual speeds for each one once they are populated. CPU should be decent and 1 CPU per GPU for Nvidia GPUs. RAM isn't too important. Storage is not too important as long as it is reliable. PSU is critical as it powers the entire build so a high wattage with highest efficiency is recommended. GPUs would be important but you need to ensure that there's enough cooling within your rig and that the GPU can fit easily with other GPUs once populated on your motherboard. In other words, a 2.5 Slot wide GPU may not work well if you have to of them side-by-side and the motherboard only has spacing for 2 slots wide GPUs.

Nuitari wrote:...The other rig is hitting the ram limit, so I'm waiting for some extra ram to come in...

AFAIK, there's no RAM increase when adding multiple GPUs into a single system that's dedicated for folding. 8 GBs or 16 GBs would be more than enough.

Nuitari · Post by **Nuitari** » Sat Apr 25, 2020 9:21 pm

PantherX wrote:
Nuitari wrote:...The other rig is hitting the ram limit, so I'm waiting for some extra ram to come in...
AFAIK, there's no RAM increase when adding multiple GPUs into a single system that's dedicated for folding. 8 GBs or 16 GBs would be more than enough.

It was to increase to 16gb

There are 7 GPUs in that rig, and some WUs are about 2.2gb in size.
Plus there is the rest of the stuff that runs on a computer including the software I use to manage the heat of the graphic card.
Even at 16gb it will barely squeak by if all WUs are of the 2gb+ size.

Post by **PantherX** » Sat Apr 25, 2020 9:27 pm

I don't have first-hand experience with 7 GPUs in a single system. However, if WUs are large, shouldn't it be stored in VRAM instead of RAM?

Post by **Joe_H** » Sat Apr 25, 2020 9:40 pm

The GPU folding core uses RAM during the setup phases for running the WU, and some during the sanity checks. In between the RAM usage shouldn't be as high. The core uses little VRAM, just the shaders.

Nuitari · Post by **Nuitari** » Sun Apr 26, 2020 12:13 am

GPU VRAM has a few hundred MB, I think the most I saw was 500mb

System ram usage stays at around 1.7gb to 2gb for projects 14561, 14562, 14563

16339 and 14549 are at ~200mb of system ram
13400 is at ~800mb

Should I file a bug report for the 2gb usage?

Folding Forum

Multi-GPU

Multi-GPU

Re: Multi-GPU

Re: Multi-GPU

Re: Multi-GPU

Re: Multi-GPU

Re: Multi-GPU

Re: Multi-GPU

Re: Multi-GPU

Re: Multi-GPU

Re: Multi-GPU

Re: Multi-GPU

Re: Multi-GPU

Re: Multi-GPU

Re: Multi-GPU

Re: Multi-GPU