How can I stop getting 13420 WUs on my GPU?

Moderators: Site Moderators, FAHC Science Team

How can I stop getting 13420 WUs on my GPU?

Postby themartymonster » Mon Aug 10, 2020 7:20 am

These take days to run on a RTX 2070 GPU.
How can I force F@H to get other WUs instead of Covid WUs?

Seems that other WU also now take an extra long time to run on GPU.
NOTE: This appears to only have started since I upgraded my PC from 32GB Ram to 64GB of Ram.

Code: Select all
********************** Log Started 2020-08-07T02:09:43Z ***********************
02:09:43:****************************** FAHClient ******************************
02:09:43:        Version: 7.6.9
02:09:43:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
02:09:43:      Copyright: 2020 foldingathome.org
02:09:43:       Homepage: https://foldingathome.org/
02:09:43:           Date: Apr 17 2020
02:09:43:           Time: 11:13:06
02:09:43:       Revision: 398c2b17fa535e0cc6c9d10856b2154c32771646
02:09:43:         Branch: master
02:09:43:       Compiler: Visual C++ 2008
02:09:43:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
02:09:43:       Platform: win32 10
02:09:43:           Bits: 32
02:09:43:           Mode: Release
02:09:43:           Args: --open-web-control
02:09:43:         Config: C:\Users\compu\AppData\Roaming\FAHClient\config.xml
02:09:43:******************************** CBang ********************************
02:09:43:           Date: Apr 17 2020
02:09:43:           Time: 11:10:09
02:09:43:       Revision: 2fb0be7809c5e45287a122ca5fbc15b5ae859a3b
02:09:43:         Branch: master
02:09:43:       Compiler: Visual C++ 2008
02:09:43:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
02:09:43:       Platform: win32 10
02:09:43:           Bits: 32
02:09:43:           Mode: Release
02:09:43:******************************* System ********************************
02:09:43:            CPU: AMD Ryzen 9 3900X 12-Core Processor
02:09:43:         CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
02:09:43:           CPUs: 24
02:09:43:         Memory: 63.92GiB
02:09:43:    Free Memory: 55.92GiB
02:09:43:        Threads: WINDOWS_THREADS
02:09:43:     OS Version: 6.2
02:09:43:    Has Battery: true
02:09:43:     On Battery: false
02:09:43:     UTC Offset: 10
02:09:43:            PID: 22492
02:09:43:            CWD: C:\Users\compu\AppData\Roaming\FAHClient
02:09:43:             OS: Windows 10 Enterprise
02:09:43:        OS Arch: AMD64
02:09:43:           GPUs: 1
02:09:43:          GPU 0: Bus:11 Slot:0 Func:0 NVIDIA:7 TU106 [GeForce RTX 2070 Rev. A] M
02:09:43:                 7465
02:09:43:  CUDA Device 0: Platform:0 Device:0 Bus:11 Slot:0 Compute:7.5 Driver:11.0
02:09:43:OpenCL Device 0: Platform:0 Device:0 Bus:11 Slot:0 Compute:1.2 Driver:451.48
02:09:43:  Win32 Service: false
02:09:43:******************************* libFAH ********************************
02:09:43:           Date: Apr 15 2020
02:09:43:           Time: 14:53:14
02:09:43:       Revision: 216968bc7025029c841ed6e36e81a03a316890d3
02:09:43:         Branch: master
02:09:43:       Compiler: Visual C++ 2008
02:09:43:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
02:09:43:       Platform: win32 10
02:09:43:           Bits: 32
02:09:43:           Mode: Release
02:09:43:***********************************************************************
Last edited by Joe_H on Mon Aug 10, 2020 2:35 pm, edited 1 time in total.
Reason: added Code tags to log
themartymonster
 
Posts: 9
Joined: Mon Apr 20, 2020 2:36 am

Re: How can I stop getting 13420 WUs on my GPU?

Postby Knish » Mon Aug 10, 2020 8:29 am

days??? I have a gtx950 and that takes 13 hrs for those WU. I've never seen your issue before so i can only suggest a complete FAH reinstall including the checkbox for removing data.
Knish
 
Posts: 92
Joined: Tue Mar 17, 2020 6:20 am

Re: How can I stop getting 13420 WUs on my GPU?

Postby gunnarre » Mon Aug 10, 2020 8:33 am

I don't think you can force it, but you can set your preference for another disease in the Cause Preference under the Advanced pane. If Covid-19 is the only work that is available, then that's what you'll get, but if you've set e.g. Cancer or Alzheimer you will get that first if any work for those are available.

That said, it sounds weird that the RTX 2070 would be that slow. I usually get between 1.3M and 2M PPD on the 13420/13421 work units on the RTX 2070 (non-super). Perhaps you could try pausing your CPU slot and see if that helps the PPD? It almost sounds like it's trying to run your work unit on the CPU or something.
Last edited by gunnarre on Mon Aug 10, 2020 9:37 am, edited 5 times in total.
Image
gunnarre
 
Posts: 170
Joined: Sun May 24, 2020 8:23 pm
Location: Norway

Re: How can I stop getting 13420 WUs on my GPU?

Postby gunnarre » Mon Aug 10, 2020 8:40 am

PS:
themartymonster wrote:NOTE: This appears to only have started since I upgraded my PC from 32GB Ram to 64GB of Ram.

Have you checked your memory timings? Are the RAM sticks of the exact same timings and speed, and inserted in the correct slots? Try to turn off XMP (D.O.C.P.) The XMP profiles that come with memory kits are matched to the number of sticks which are in the kit. If you put two kits together, then you might need to run the memory a bit slower than you used to.
gunnarre
 
Posts: 170
Joined: Sun May 24, 2020 8:23 pm
Location: Norway

Re: How can I stop getting 13420 WUs on my GPU?

Postby ChristianVirtual » Mon Aug 10, 2020 9:01 am

Can you please also share the slot setup ? Wonder if you CPU might be overallocated and don’t have a CPU thread for the GPU always around ? Not sure if still possible these days
ImageImage
Please contribute your logs to http://ppd.fahmm.net
User avatar
ChristianVirtual
 
Posts: 1596
Joined: Tue May 28, 2013 1:14 pm
Location: Tokyo

Re: How can I stop getting 13420 WUs on my GPU?

Postby marknd59 » Mon Aug 10, 2020 12:41 pm

Some strange is going on with 13420 WUs. I've have a range of PPD with them that goes from 600K up to 1.5M with different WUs.
Image
marknd59
 
Posts: 22
Joined: Tue Apr 28, 2020 9:05 am

Re: How can I stop getting 13420 WUs on my GPU?

Postby gunnarre » Mon Aug 10, 2020 12:46 pm

The researchers are are aware of the variability, and are doing some testing on it. They're also working on distributing WUs to better matched GPUs/systems, and that might involve the FAH client running short benchmark on the system, or at least changing the assigments a bit.

In the mean time, they've increased the baseline points for 13420 and 13421 WUs, to try to compensate for the variability.
gunnarre
 
Posts: 170
Joined: Sun May 24, 2020 8:23 pm
Location: Norway

Re: How can I stop getting 13420 WUs on my GPU?

Postby bruce » Mon Aug 10, 2020 5:22 pm

themartymonster wrote:NOTE: This appears to only have started since I upgraded my PC from 32GB Ram to 64GB of Ram.

While it's possible the change in RAM is related, I think it's very unlikely. I suspect it's simply a temporary change in the priority of p13420 compared to changes to other COVID19 project.
bruce
 
Posts: 19970
Joined: Thu Nov 29, 2007 11:13 pm
Location: So. Cal.

Re: How can I stop getting 13420 WUs on my GPU?

Postby HaloJones » Mon Aug 10, 2020 9:00 pm

I'm having no issues with 13420 on my Maxwell and Pascal GPU. In fact quite the opposite at the moment with even my worst performing 1070 getting over 1m ppd.
1x Titan X, 5x 1070, 1x 970, 1 x Ryzen 3600

Image
HaloJones
 
Posts: 868
Joined: Thu Jul 24, 2008 11:16 am

Re: How can I stop getting 13420 WUs on my GPU?

Postby themartymonster » Tue Aug 11, 2020 12:09 am

Thanks for all of the replies.
All 4 Memory cards are the same brand, type etc.
4 x 16GB = 64GB
CPU AMD 3900X
GPU ASUS 2070

CPU usage was less than 70%

I stopped running everything this morning and did ANOTHER reset and also set the priority to Alzheimers and now it is running
Work Unit (PRCG) 16918 (12, 49, 16) Work Unit (ETA) 2 hours 58 mins 183291 Estimated Points
1401446 Points per day

Will see how it goes.
themartymonster
 
Posts: 9
Joined: Mon Apr 20, 2020 2:36 am

Re: How can I stop getting 13420 WUs on my GPU?

Postby themartymonster » Tue Aug 11, 2020 1:42 am

Okay, found the problem.
GPU is stuck at 300MHz.
Turn on the PC and the GPU fans spin up for a few seconds and then stop spinning.
The the GPU will throttle its speed at 300MHz.
Now to see if it is the power supply or GPU which is causing it.

And yes, I took out the extra 2 RAM cards and it did not make any difference.
themartymonster
 
Posts: 9
Joined: Mon Apr 20, 2020 2:36 am

Re: How can I stop getting 13420 WUs on my GPU?

Postby uyaem » Tue Aug 11, 2020 6:30 am

If the temperatures on the GPU are within the normal range, and it is throttling that much, it would seem likely that it's GPU or driver related.
Have you updated your drivers recently?
You could also try and re-install/repair those, Windows updates have the habit to sometimes break the part of it that is needed for FAH.
Image
CPU: Ryzen 9 3900X (1x21 CPUs) ~ GPU: nVidia GeForce GTX 1660 Super (Asus)
uyaem
 
Posts: 222
Joined: Sat Mar 21, 2020 8:35 pm
Location: Esslingen, Germany

Re: How can I stop getting 13420 WUs on my GPU?

Postby themartymonster » Tue Aug 11, 2020 7:54 am

uyaem wrote:You could also try and re-install/repair those, Windows updates have the habit to sometimes break the part of it that is needed for FAH.


I took it out and put it in another spare PC which has an AMD GPU.
Took out AMD GPU, put this NVIDIA GPU in and powered it on.
Ran a GPU bench with Hwinfo64 and it worked just like it should.
Put it back in the original PC but used a different PCIE power cable, by that, the power supply has a few different PCIE power cables, some hardwired and the others a plug in to power supply.
I used one that was hardwired instead of the plug in cable.
Ran the benchmark and it is working as it should.
Problem fixed.

Power Supply is a Corsair HX1000 which I have had for a few years.
themartymonster
 
Posts: 9
Joined: Mon Apr 20, 2020 2:36 am

Re: How can I stop getting 13420 WUs on my GPU?

Postby Neil-B » Tue Aug 11, 2020 8:02 am

Really glad you finally got it sorted :)
1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent, Quadro K420 1GB, FAH 7.6.13
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro, Quadro M1000M 2GB, FAH 7.6.13
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro, GTX 750Ti 2GB, FAH 7.6.13
Neil-B
 
Posts: 1405
Joined: Sun Mar 22, 2020 6:52 pm
Location: UK

Re: How can I stop getting 13420 WUs on my GPU?

Postby gunnarre » Tue Aug 11, 2020 10:55 am

Hooray. Remember to set your cause preference back to "Any".
gunnarre
 
Posts: 170
Joined: Sun May 24, 2020 8:23 pm
Location: Norway

Next

Return to Issues with a specific WU

Who is online

Users browsing this forum: midhart90 and 2 guests

cron