Folding Forum

RTX 2070 is in top PCIe slot for direct connect to CPU
GTX 1650 is in second slot. I chose this card because it does not take any PSU power, it is limited to 75w coming directly from the PCIe slot.

I used nvidia-smi to limit the 2070 to 160w from the card limit of 180w, and the temp stays right in the middle 70s C

The combination is delivering a steady 2.0 million PPD with small power and low temps.

https://imgur.com/Ugqsd3w

pcwolf wrote:RTX 2070 is in top PCIe slot for direct connect to CPU
GTX 1650 is in second slot. I chose this card because it does not take any PSU power, it is limited to 75w coming directly from the PCIe slot.

I used nvidia-smi to limit the 2070 to 160w from the card limit of 180w, and the temp stays right in the middle 70s C

The combination is delivering a steady 2.0 million PPD with small power and low temps.

https://imgur.com/Ugqsd3w

Interesting. I was getting 1.1-1.2 million out of one vanilla GTX 2060. With a second added, it looks like 1.6-1.7 million. Not sure it was worth it. Maybe points don't quite reflect the actual amount of computation being done, and maybe I'm getting nearer twice the work done.

The cards are just over the power supply and almost adjacent. One runs around 76 C and the other a bit cooler (60 to 70 C usually) despite a lot of possibly misplaced fans. Seems to be some suppression -- sometimes one card is slowed down a bit presumably to prevent overheating.

Just saw that when a card went idle for a few minutes its temperature dropped quickly to around 30 C, so the cooling can't be complete rubbish...

something not right there then. take a look at Advanced Control. Which card is getting what work? clicking on each slot shows you the estimated ppd for that unit. is it consistent which card is getting the points?

are you folding on the cpu as well? make sure there is one thread for each gpu slot.

download GPU-Z and see if both cards are running at full load or is one significantly faster/slower/throttling compared to the other.

IME, two identical cards will get on average the same number of points unless there is a constraint on one of them. constraints could include desktop use, PCIE slot speed, heat, boost speeds, slot configuration.

HaloJones wrote:something not right there then. take a look at Advanced Control. Which card is getting what work? clicking on each slot shows you the estimated ppd for that unit. is it consistent which card is getting the points?

are you folding on the cpu as well? make sure there is one thread for each gpu slot.

download GPU-Z and see if both cards are running at full load or is one significantly faster/slower/throttling compared to the other.

IME, two identical cards will get on average the same number of points unless there is a constraint on one of them. constraints could include desktop use, PCIE slot speed, heat, boost speeds, slot configuration.

Thanks, will take a look. Fewer tools maybe on Linux but there are still some.

nvidia server settings may let you see various important data points:

PCIE speed, PCIE utilisation, gpu core speed

from the CLI nvidia-smi allows you to see power usage (W) and utilisation rate (%)

With the two vanilla RTX 2060s running (one around 60-65C, other 70-75C usually), and the CPU working 10 threads, the momentary PPD estimate displayed on FAHControl is often over 2 million. Not always. I think I'm more or less getting my money's worth now. But it varies a lot.

About US$0.10 per kWh here but the next power bill will be... interesting. Still plan to leave both cards working if I can.

I asked this already on the New Folders forum, so sorry for the repetition.

EDIT BY Mod:
Merged your other topic and this one.


I have an RTX 2060 KO (TU 106-200A) in what seems to the the principal slot of the motherboard. GPU Utilization is usually 95% or higher (all figures are from the NVIDIA X Servers Settings application on Ubuntu). Bus Type: PCI Express x16 Gen3. Maximum width of the PCIe link is x16 and maximum speed is 8.0 GT/s, with utilization usually under 40%. Card runs hot, up to 77C but usually a little over 70C. It's physically higher up in the case.

I also have an RTX 2060 (TU 106) in the other wide slot of the motherboard. GPU Utilization is always lower than on the other card, sometimes under 70%, sometimes near 90%. Bus Type: PCI Express x16 Gen2. PCIe link maxima are x16 and 5.0 GT/s, utilization usually around 70%. This card is lower in the case and runs cooler, usually 65C but sometimes as high as 73C.

So why does the second card always seem to have lower GPU Utilization? is it related to the lower bandwidth of the bus? The bandwidth still seems sufficient, just lower.

It's not a big deal, just that I'd like to get as much as I can out of this hardware, and I'm puzzled. The instantaneous PPD estimates displayed by FAHControl vary from 1.4M to 2.2M, often near 2M. I've been folding, running 10 threads, on the 6-core Ryzen 5 1600AF CPU as well, contributing a little to that, but currently I've shut off folding on the CPU to see if it that makes any difference with the GPUs (I suspect it won't matter: so far it doesn't seem to).

Any ideas? Thanks.

That's not too bad of a usage. In Linux, the bus bandwidth should not be limiting those cards. I'd suspect the CPU cores are saturated...running ten threads on a 6-core might be limiting the GPUs. I tend to leave one real core per GPU, so that would mean running 4 threads on the CPU (it will map them to real cores). The other two cores can feed the GPUs.

1- Make sure you have at least 2 CPU cores saved for the GPUs (don't fold on all your CPU cores).
2- Make sure your CPU runs at least 2,5 Ghz (3 Ghz preferred) or faster.
3- Your double x16 slots may be configured in a different way. Either your BIOS, GPUZ, or the Nvidia driver software can tell you what bus speed your second GPU is running at.
While they both may fit an x16 slot on the motherboard,
The top one is probably running at x8 speed, and the bottom one could be running at x8, x4, or x1 speed.
If that's PCIE2.0, it would equal running the GPU at PCIE 3.0 x4, x2, or x0.5 speeds. The latter two would bottleneck the 2060.

Your best bet is to run Linux, and fold from that.
If your bottom port is PCIE 2.0 x8, it wouldn't bottleneck the 2060 in Linux, but might in Windows.

I’ll see what the BIOS settings can do about the PCIe slots. Keeping the 6-core CPU to only 4 cores of folding (currently it’s doing none) sounds like a good idea. This processor has a base clock of 3.2 GHz which I think should be fast enough.

Thanks to everyone for suggestions.

GPU-Z should show the max and used PCI lanes

https://www.techpowerup.com/gpuz/

Titled Bus Interface on the main screen, It may well be different for each GPU.

Is GPU-Z only for Windows? I’m running Ubuntu.

markdotgooley wrote:Is GPU-Z only for Windows? I’m running Ubuntu.

Ah, If you had posted your log including the configuration section, we would know this, as it is, I am recommending utilities that won't work and MeeLee is recommending you switch to an OS you already use.

Linux has much less an issue with PCIE bandwidth, but I am not a user so can't recomend any utilities to see if you are suffering from it.

viewtopic.php?f=24&t=26036

You can check your PCIE x8/x4 speed in Nvidia-Xserver (the Nvidia panel found in preferences),

Found out one reason why one RTX 2060 in my computer usually has GPU utilization over 95% and the other often has GPU utilization around 75%. It’s a quirk of the motherboard. There are two wide PCIe slot connectors that suit a x16 connection. The main one provides that. The second one provides an x4 connection, unless any sort of device is plugged into one of the little x1 connectors. In that case, the second wide slot provides not an x4 connection but an x1.

My folding machine uses a PCIe x1 WiFi card to communicate, so the second 2060 gets an x1 connection and I’m guessing that this causes the lower GPU utilization.

I’m going to try using a USB dongle for WiFi instead. We’ll see if this helps.