How can I stop getting 13420 WUs on my GPU?

Moderators: Site Moderators, FAHC Science Team

themartymonster
Posts: 9
Joined: Mon Apr 20, 2020 1:36 am

How can I stop getting 13420 WUs on my GPU?

Post by themartymonster »

These take days to run on a RTX 2070 GPU.
How can I force F@H to get other WUs instead of Covid WUs?

Seems that other WU also now take an extra long time to run on GPU.
NOTE: This appears to only have started since I upgraded my PC from 32GB Ram to 64GB of Ram.

Code: Select all

********************** Log Started 2020-08-07T02:09:43Z ***********************
02:09:43:****************************** FAHClient ******************************
02:09:43:        Version: 7.6.9
02:09:43:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
02:09:43:      Copyright: 2020 foldingathome.org
02:09:43:       Homepage: https://foldingathome.org/
02:09:43:           Date: Apr 17 2020
02:09:43:           Time: 11:13:06
02:09:43:       Revision: 398c2b17fa535e0cc6c9d10856b2154c32771646
02:09:43:         Branch: master
02:09:43:       Compiler: Visual C++ 2008
02:09:43:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
02:09:43:       Platform: win32 10
02:09:43:           Bits: 32
02:09:43:           Mode: Release
02:09:43:           Args: --open-web-control
02:09:43:         Config: C:\Users\compu\AppData\Roaming\FAHClient\config.xml
02:09:43:******************************** CBang ********************************
02:09:43:           Date: Apr 17 2020
02:09:43:           Time: 11:10:09
02:09:43:       Revision: 2fb0be7809c5e45287a122ca5fbc15b5ae859a3b
02:09:43:         Branch: master
02:09:43:       Compiler: Visual C++ 2008
02:09:43:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
02:09:43:       Platform: win32 10
02:09:43:           Bits: 32
02:09:43:           Mode: Release
02:09:43:******************************* System ********************************
02:09:43:            CPU: AMD Ryzen 9 3900X 12-Core Processor
02:09:43:         CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
02:09:43:           CPUs: 24
02:09:43:         Memory: 63.92GiB
02:09:43:    Free Memory: 55.92GiB
02:09:43:        Threads: WINDOWS_THREADS
02:09:43:     OS Version: 6.2
02:09:43:    Has Battery: true
02:09:43:     On Battery: false
02:09:43:     UTC Offset: 10
02:09:43:            PID: 22492
02:09:43:            CWD: C:\Users\compu\AppData\Roaming\FAHClient
02:09:43:             OS: Windows 10 Enterprise
02:09:43:        OS Arch: AMD64
02:09:43:           GPUs: 1
02:09:43:          GPU 0: Bus:11 Slot:0 Func:0 NVIDIA:7 TU106 [GeForce RTX 2070 Rev. A] M
02:09:43:                 7465
02:09:43:  CUDA Device 0: Platform:0 Device:0 Bus:11 Slot:0 Compute:7.5 Driver:11.0
02:09:43:OpenCL Device 0: Platform:0 Device:0 Bus:11 Slot:0 Compute:1.2 Driver:451.48
02:09:43:  Win32 Service: false
02:09:43:******************************* libFAH ********************************
02:09:43:           Date: Apr 15 2020
02:09:43:           Time: 14:53:14
02:09:43:       Revision: 216968bc7025029c841ed6e36e81a03a316890d3
02:09:43:         Branch: master
02:09:43:       Compiler: Visual C++ 2008
02:09:43:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
02:09:43:       Platform: win32 10
02:09:43:           Bits: 32
02:09:43:           Mode: Release
02:09:43:***********************************************************************
Last edited by Joe_H on Mon Aug 10, 2020 1:35 pm, edited 1 time in total.
Reason: added Code tags to log
Knish
Posts: 232
Joined: Tue Mar 17, 2020 5:20 am

Re: How can I stop getting 13420 WUs on my GPU?

Post by Knish »

days??? I have a gtx950 and that takes 13 hrs for those WU. I've never seen your issue before so i can only suggest a complete FAH reinstall including the checkbox for removing data.
gunnarre
Posts: 567
Joined: Sun May 24, 2020 7:23 pm
Location: Norway

Re: How can I stop getting 13420 WUs on my GPU?

Post by gunnarre »

I don't think you can force it, but you can set your preference for another disease in the Cause Preference under the Advanced pane. If Covid-19 is the only work that is available, then that's what you'll get, but if you've set e.g. Cancer or Alzheimer you will get that first if any work for those are available.

That said, it sounds weird that the RTX 2070 would be that slow. I usually get between 1.3M and 2M PPD on the 13420/13421 work units on the RTX 2070 (non-super). Perhaps you could try pausing your CPU slot and see if that helps the PPD? It almost sounds like it's trying to run your work unit on the CPU or something.
Last edited by gunnarre on Mon Aug 10, 2020 8:37 am, edited 5 times in total.
Image
Online: GTX 1660 Super, GTX 1080, GTX 1050 Ti 4G OC, RX580 + occasional CPU folding in the cold.
Offline: Radeon HD 7770, GTX 960, GTX 950
gunnarre
Posts: 567
Joined: Sun May 24, 2020 7:23 pm
Location: Norway

Re: How can I stop getting 13420 WUs on my GPU?

Post by gunnarre »

PS:
themartymonster wrote: NOTE: This appears to only have started since I upgraded my PC from 32GB Ram to 64GB of Ram.
Have you checked your memory timings? Are the RAM sticks of the exact same timings and speed, and inserted in the correct slots? Try to turn off XMP (D.O.C.P.) The XMP profiles that come with memory kits are matched to the number of sticks which are in the kit. If you put two kits together, then you might need to run the memory a bit slower than you used to.
Image
Online: GTX 1660 Super, GTX 1080, GTX 1050 Ti 4G OC, RX580 + occasional CPU folding in the cold.
Offline: Radeon HD 7770, GTX 960, GTX 950
ChristianVirtual
Posts: 1596
Joined: Tue May 28, 2013 12:14 pm
Location: Tokyo

Re: How can I stop getting 13420 WUs on my GPU?

Post by ChristianVirtual »

Can you please also share the slot setup ? Wonder if you CPU might be overallocated and don’t have a CPU thread for the GPU always around ? Not sure if still possible these days
ImageImage
Please contribute your logs to http://ppd.fahmm.net
marknd59
Posts: 22
Joined: Tue Apr 28, 2020 8:05 am

Re: How can I stop getting 13420 WUs on my GPU?

Post by marknd59 »

Some strange is going on with 13420 WUs. I've have a range of PPD with them that goes from 600K up to 1.5M with different WUs.
Image
gunnarre
Posts: 567
Joined: Sun May 24, 2020 7:23 pm
Location: Norway

Re: How can I stop getting 13420 WUs on my GPU?

Post by gunnarre »

The researchers are are aware of the variability, and are doing some testing on it. They're also working on distributing WUs to better matched GPUs/systems, and that might involve the FAH client running short benchmark on the system, or at least changing the assigments a bit.

In the mean time, they've increased the baseline points for 13420 and 13421 WUs, to try to compensate for the variability.
Image
Online: GTX 1660 Super, GTX 1080, GTX 1050 Ti 4G OC, RX580 + occasional CPU folding in the cold.
Offline: Radeon HD 7770, GTX 960, GTX 950
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: How can I stop getting 13420 WUs on my GPU?

Post by bruce »

themartymonster wrote:NOTE: This appears to only have started since I upgraded my PC from 32GB Ram to 64GB of Ram.
While it's possible the change in RAM is related, I think it's very unlikely. I suspect it's simply a temporary change in the priority of p13420 compared to changes to other COVID19 project.
HaloJones
Posts: 920
Joined: Thu Jul 24, 2008 10:16 am

Re: How can I stop getting 13420 WUs on my GPU?

Post by HaloJones »

I'm having no issues with 13420 on my Maxwell and Pascal GPU. In fact quite the opposite at the moment with even my worst performing 1070 getting over 1m ppd.
single 1070

Image
themartymonster
Posts: 9
Joined: Mon Apr 20, 2020 1:36 am

Re: How can I stop getting 13420 WUs on my GPU?

Post by themartymonster »

Thanks for all of the replies.
All 4 Memory cards are the same brand, type etc.
4 x 16GB = 64GB
CPU AMD 3900X
GPU ASUS 2070

CPU usage was less than 70%

I stopped running everything this morning and did ANOTHER reset and also set the priority to Alzheimers and now it is running
Work Unit (PRCG) 16918 (12, 49, 16) Work Unit (ETA) 2 hours 58 mins 183291 Estimated Points
1401446 Points per day

Will see how it goes.
themartymonster
Posts: 9
Joined: Mon Apr 20, 2020 1:36 am

Re: How can I stop getting 13420 WUs on my GPU?

Post by themartymonster »

Okay, found the problem.
GPU is stuck at 300MHz.
Turn on the PC and the GPU fans spin up for a few seconds and then stop spinning.
The the GPU will throttle its speed at 300MHz.
Now to see if it is the power supply or GPU which is causing it.

And yes, I took out the extra 2 RAM cards and it did not make any difference.
uyaem
Posts: 222
Joined: Sat Mar 21, 2020 7:35 pm
Location: Esslingen, Germany

Re: How can I stop getting 13420 WUs on my GPU?

Post by uyaem »

If the temperatures on the GPU are within the normal range, and it is throttling that much, it would seem likely that it's GPU or driver related.
Have you updated your drivers recently?
You could also try and re-install/repair those, Windows updates have the habit to sometimes break the part of it that is needed for FAH.
Image
CPU: Ryzen 9 3900X (1x21 CPUs) ~ GPU: nVidia GeForce GTX 1660 Super (Asus)
themartymonster
Posts: 9
Joined: Mon Apr 20, 2020 1:36 am

Re: How can I stop getting 13420 WUs on my GPU?

Post by themartymonster »

uyaem wrote: You could also try and re-install/repair those, Windows updates have the habit to sometimes break the part of it that is needed for FAH.
I took it out and put it in another spare PC which has an AMD GPU.
Took out AMD GPU, put this NVIDIA GPU in and powered it on.
Ran a GPU bench with Hwinfo64 and it worked just like it should.
Put it back in the original PC but used a different PCIE power cable, by that, the power supply has a few different PCIE power cables, some hardwired and the others a plug in to power supply.
I used one that was hardwired instead of the plug in cable.
Ran the benchmark and it is working as it should.
Problem fixed.

Power Supply is a Corsair HX1000 which I have had for a few years.
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: How can I stop getting 13420 WUs on my GPU?

Post by Neil-B »

Really glad you finally got it sorted :)
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
gunnarre
Posts: 567
Joined: Sun May 24, 2020 7:23 pm
Location: Norway

Re: How can I stop getting 13420 WUs on my GPU?

Post by gunnarre »

Hooray. Remember to set your cause preference back to "Any".
Image
Online: GTX 1660 Super, GTX 1080, GTX 1050 Ti 4G OC, RX580 + occasional CPU folding in the cold.
Offline: Radeon HD 7770, GTX 960, GTX 950
Post Reply