System freezes when folding is started

[Petr_Z]

Code: Select all

*********************** Log Started 2020-04-20T20:24:53Z ***********************
20:24:53:****************************** FAHClient ******************************
20:24:53:        Version: 7.6.9
20:24:53:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
20:24:53:      Copyright: 2020 foldingathome.org
20:24:53:       Homepage: https://foldingathome.org/
20:24:53:           Date: Apr 17 2020
20:24:53:           Time: 11:13:06
20:24:53:       Revision: 398c2b17fa535e0cc6c9d10856b2154c32771646
20:24:53:         Branch: master
20:24:53:       Compiler: Visual C++ 2008
20:24:53:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
20:24:53:       Platform: win32 10
20:24:53:           Bits: 32
20:24:53:           Mode: Release
20:24:53:         Config: C:\Users\petrz\AppData\Roaming\FAHClient\config.xml
20:24:53:******************************** CBang ********************************
20:24:53:           Date: Apr 17 2020
20:24:53:           Time: 11:10:09
20:24:53:       Revision: 2fb0be7809c5e45287a122ca5fbc15b5ae859a3b
20:24:53:         Branch: master
20:24:53:       Compiler: Visual C++ 2008
20:24:53:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
20:24:53:       Platform: win32 10
20:24:53:           Bits: 32
20:24:53:           Mode: Release
20:24:53:******************************* System ********************************
20:24:53:            CPU: AMD Ryzen 9 3900X 12-Core Processor
20:24:53:         CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
20:24:53:           CPUs: 24
20:24:53:         Memory: 31.91GiB
20:24:53:    Free Memory: 28.43GiB
20:24:53:        Threads: WINDOWS_THREADS
20:24:53:     OS Version: 6.2
20:24:53:    Has Battery: false
20:24:53:     On Battery: false
20:24:53:     UTC Offset: 2
20:24:53:            PID: 12328
20:24:53:            CWD: C:\Users\petrz\AppData\Roaming\FAHClient
20:24:53:             OS: Windows 10 Enterprise
20:24:53:        OS Arch: AMD64
20:24:53:           GPUs: 2
20:24:53:          GPU 0: Bus:14 Slot:0 Func:0 AMD:5 Vega 10 XL/XT [Radeon RX Vega 56/64]
20:24:53:          GPU 1: Bus:11 Slot:0 Func:0 AMD:5 Vega 10 XL/XT [Radeon RX Vega 56/64]
20:24:53:           CUDA: Not detected: Failed to open dynamic library 'nvcuda.dll':
20:24:53:                 Uvedený modul nebyl nalezen.
20:24:53:
20:24:53:OpenCL Device 0: Platform:0 Device:0 Bus:11 Slot:0 Compute:1.2 Driver:3004.7
20:24:53:OpenCL Device 1: Platform:0 Device:1 Bus:14 Slot:0 Compute:1.2 Driver:3004.7
20:24:53:  Win32 Service: false
20:24:53:******************************* libFAH ********************************
20:24:53:           Date: Apr 15 2020
20:24:53:           Time: 14:53:14
20:24:53:       Revision: 216968bc7025029c841ed6e36e81a03a316890d3
20:24:53:         Branch: master
20:24:53:       Compiler: Visual C++ 2008
20:24:53:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
20:24:53:       Platform: win32 10
20:24:53:           Bits: 32
20:24:53:           Mode: Release
20:24:53:***********************************************************************
20:24:53:<config>
20:24:53:  <!-- Network -->
20:24:53:  <proxy v=':8080'/>
20:24:53:
20:24:53:  <!-- Slot Control -->
20:24:53:  <power v='FULL'/>
20:24:53:
20:24:53:  <!-- User Information -->
20:24:53:  <passkey v='*****'/>
20:24:53:  <user v='Petr_Z'/>
20:24:53:
20:24:53:  <!-- Folding Slots -->
20:24:53:  <slot id='0' type='CPU'/>
20:24:53:  <slot id='1' type='GPU'/>
20:24:53:  <slot id='2' type='GPU'/>
20:24:53:</config>
20:24:53:Trying to access database...
20:24:53:Successfully acquired database lock
20:24:53:Enabled folding slot 00: READY cpu:22
20:24:53:Enabled folding slot 01: READY gpu:0:Vega 10 XL/XT [Radeon RX Vega 56/64]
20:24:53:Enabled folding slot 02: READY gpu:1:Vega 10 XL/XT [Radeon RX Vega 56/64]
20:24:53:WU01:FS02:Starting
20:24:53:WU01:FS02:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\petrz\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/v7/win/64bit/Core_22.fah/FahCore_22.exe -dir 01 -suffix 01 -version 706 -lifeline 12328 -checkpoint 15 -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
20:24:53:WU01:FS02:Started FahCore on PID 12432
20:24:53:WU01:FS02:Core PID:12456
20:24:53:WU01:FS02:FahCore 0x22 started
20:24:53:WU01:FS02:0x22:*********************** Log Started 2020-04-20T20:24:53Z ***********************
20:24:53:WU01:FS02:0x22:*************************** Core22 Folding@home Core ***************************
20:24:53:WU01:FS02:0x22:       Type: 0x22
20:24:53:WU01:FS02:0x22:       Core: Core22
20:24:53:WU01:FS02:0x22:    Website: https://foldingathome.org/
20:24:53:WU01:FS02:0x22:  Copyright: (c) 2009-2018 foldingathome.org
20:24:53:WU01:FS02:0x22:     Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
20:24:53:WU01:FS02:0x22:             <rafal.wiewiora@choderalab.org>
20:24:53:WU01:FS02:0x22:       Args: -dir 01 -suffix 01 -version 706 -lifeline 12432 -checkpoint 15
20:24:53:WU01:FS02:0x22:             -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
20:24:53:WU01:FS02:0x22:     Config: <none>
20:24:53:WU01:FS02:0x22:************************************ Build *************************************
20:24:53:WU01:FS02:0x22:    Version: 0.0.2
20:24:53:WU01:FS02:0x22:       Date: Dec 6 2019
20:24:53:WU01:FS02:0x22:       Time: 21:30:31
20:24:53:WU01:FS02:0x22: Repository: Git
20:24:53:WU01:FS02:0x22:   Revision: abeb39247cc72df5af0f63723edafadb23d5dfbe
20:24:53:WU01:FS02:0x22:     Branch: HEAD
20:24:53:WU01:FS02:0x22:   Compiler: Visual C++ 2008
20:24:53:WU01:FS02:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
20:24:53:WU01:FS02:0x22:   Platform: win32 10
20:24:53:WU01:FS02:0x22:       Bits: 64
20:24:53:WU01:FS02:0x22:       Mode: Release
20:24:53:WU01:FS02:0x22:************************************ System ************************************
20:24:53:WU01:FS02:0x22:        CPU: AMD Ryzen 9 3900X 12-Core Processor
20:24:53:WU01:FS02:0x22:     CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
20:24:53:WU01:FS02:0x22:       CPUs: 24
20:24:53:WU01:FS02:0x22:     Memory: 31.91GiB
20:24:53:WU01:FS02:0x22:Free Memory: 28.41GiB
20:24:53:WU01:FS02:0x22:    Threads: WINDOWS_THREADS
20:24:53:WU01:FS02:0x22: OS Version: 6.2
20:24:53:WU01:FS02:0x22:Has Battery: false
20:24:53:WU01:FS02:0x22: On Battery: false
20:24:53:WU01:FS02:0x22: UTC Offset: 2
20:24:53:WU01:FS02:0x22:        PID: 12456
20:24:53:WU01:FS02:0x22:        CWD: C:\Users\petrz\AppData\Roaming\FAHClient\work
20:24:53:WU01:FS02:0x22:         OS: Windows 10 Pro
20:24:53:WU01:FS02:0x22:    OS Arch: AMD64
20:24:53:WU01:FS02:0x22:********************************************************************************
20:24:53:WU01:FS02:0x22:Project: 11763 (Run 0, Clone 5192, Gen 44)
20:24:53:WU01:FS02:0x22:Unit: 0x0000005080fccb0a5e71137eea98fdb2
20:24:53:WU01:FS02:0x22:Digital signatures verified
20:24:53:WU01:FS02:0x22:Folding@home GPU Core22 Folding@home Core
20:24:53:WU01:FS02:0x22:Version 0.0.2
20:24:54:WU02:FS01:Starting
20:24:54:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\petrz\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/v7/win/64bit/Core_22.fah/FahCore_22.exe -dir 02 -suffix 01 -version 706 -lifeline 12328 -checkpoint 15 -gpu-vendor amd -opencl-platform 0 -opencl-device 1 -gpu 1
20:24:54:WU02:FS01:Started FahCore on PID 12500
20:24:54:WU02:FS01:Core PID:12524
20:24:54:WU02:FS01:FahCore 0x22 started
20:24:54:WU00:FS00:Connecting to 65.254.110.245:80
20:24:54:WU02:FS01:0x22:*********************** Log Started 2020-04-20T20:24:54Z ***********************
20:24:54:WU02:FS01:0x22:*************************** Core22 Folding@home Core ***************************
20:24:54:WU02:FS01:0x22:       Type: 0x22
20:24:54:WU02:FS01:0x22:       Core: Core22
20:24:54:WU02:FS01:0x22:    Website: https://foldingathome.org/
20:24:54:WU02:FS01:0x22:  Copyright: (c) 2009-2018 foldingathome.org
20:24:54:WU02:FS01:0x22:     Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
20:24:54:WU02:FS01:0x22:             <rafal.wiewiora@choderalab.org>
20:24:54:WU02:FS01:0x22:       Args: -dir 02 -suffix 01 -version 706 -lifeline 12500 -checkpoint 15
20:24:54:WU02:FS01:0x22:             -gpu-vendor amd -opencl-platform 0 -opencl-device 1 -gpu 1
20:24:54:WU02:FS01:0x22:     Config: <none>
20:24:54:WU02:FS01:0x22:************************************ Build *************************************
20:24:54:WU02:FS01:0x22:    Version: 0.0.2
20:24:54:WU02:FS01:0x22:       Date: Dec 6 2019
20:24:54:WU02:FS01:0x22:       Time: 21:30:31
20:24:54:WU02:FS01:0x22: Repository: Git
20:24:54:WU02:FS01:0x22:   Revision: abeb39247cc72df5af0f63723edafadb23d5dfbe
20:24:54:WU02:FS01:0x22:     Branch: HEAD
20:24:54:WU02:FS01:0x22:   Compiler: Visual C++ 2008
20:24:54:WU02:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
20:24:54:WU02:FS01:0x22:   Platform: win32 10
20:24:54:WU02:FS01:0x22:       Bits: 64
20:24:54:WU02:FS01:0x22:       Mode: Release
20:24:54:WU02:FS01:0x22:************************************ System ************************************
20:24:54:WU02:FS01:0x22:        CPU: AMD Ryzen 9 3900X 12-Core Processor
20:24:54:WU02:FS01:0x22:     CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
20:24:54:WU02:FS01:0x22:       CPUs: 24
20:24:54:WU02:FS01:0x22:     Memory: 31.91GiB
20:24:54:WU02:FS01:0x22:Free Memory: 28.36GiB
20:24:54:WU02:FS01:0x22:    Threads: WINDOWS_THREADS
20:24:54:WU02:FS01:0x22: OS Version: 6.2
20:24:54:WU02:FS01:0x22:Has Battery: false
20:24:54:WU02:FS01:0x22: On Battery: false
20:24:54:WU02:FS01:0x22: UTC Offset: 2
20:24:54:WU02:FS01:0x22:        PID: 12524
20:24:54:WU02:FS01:0x22:        CWD: C:\Users\petrz\AppData\Roaming\FAHClient\work
20:24:54:WU02:FS01:0x22:         OS: Windows 10 Pro
20:24:54:WU02:FS01:0x22:    OS Arch: AMD64
20:24:54:WU02:FS01:0x22:********************************************************************************
20:24:54:WU02:FS01:0x22:Project: 14417 (Run 0, Clone 1531, Gen 12)
20:24:54:WU02:FS01:0x22:Unit: 0x000000140d5262775e84c8a50152cd5b
20:24:54:WU02:FS01:0x22:Digital signatures verified
20:24:54:WU02:FS01:0x22:Folding@home GPU Core22 Folding@home Core
20:24:54:WU02:FS01:0x22:Version 0.0.2
20:24:55:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
20:24:55:WU00:FS00:Connecting to 18.218.241.186:80
20:24:56:WARNING:WU00:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
20:24:56:WU00:FS00:Connecting to 65.254.110.245:80
20:24:57:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
20:24:57:WU00:FS00:Connecting to 18.218.241.186:80
20:24:58:WARNING:WU00:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
20:24:58:ERROR:WU00:FS00:Exception: Could not get an assignment
20:24:58:WU00:FS00:Connecting to 65.254.110.245:80
20:24:58:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
20:24:58:WU00:FS00:Connecting to 18.218.241.186:80
20:24:59:WARNING:WU00:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
20:24:59:WU00:FS00:Connecting to 65.254.110.245:80
20:24:59:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
20:24:59:WU00:FS00:Connecting to 18.218.241.186:80
20:25:00:WARNING:WU00:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
20:25:00:ERROR:WU00:FS00:Exception: Could not get an assignment
20:25:06:WU01:FS02:0x22:Completed 0 out of 1000000 steps (0%)
20:25:06:WU01:FS02:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
20:25:18:WU02:FS01:0x22:Completed 0 out of 1000000 steps (0%)
20:25:18:WU02:FS01:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
20:25:41:3:127.0.0.1:New Web session
20:25:58:WU00:FS00:Connecting to 65.254.110.245:80
20:25:58:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
20:25:58:WU00:FS00:Connecting to 18.218.241.186:80
20:25:59:WARNING:WU00:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
20:25:59:WU00:FS00:Connecting to 65.254.110.245:80
20:25:59:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
20:25:59:WU00:FS00:Connecting to 18.218.241.186:80
20:26:00:WARNING:WU00:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
20:26:00:ERROR:WU00:FS00:Exception: Could not get an assignment
20:26:01:WU01:FS02:0x22:Completed 10000 out of 1000000 steps (1%)
20:26:54:WU01:FS02:0x22:Completed 20000 out of 1000000 steps (2%)
20:27:35:WU00:FS00:Connecting to 65.254.110.245:80
20:27:36:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
20:27:36:WU00:FS00:Connecting to 18.218.241.186:80
20:27:36:WARNING:WU00:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
20:27:36:WU00:FS00:Connecting to 65.254.110.245:80
20:27:37:WARNING:WU00:FS00:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
20:27:37:WU00:FS00:Connecting to 18.218.241.186:80
20:27:37:WARNING:WU00:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
20:27:37:ERROR:WU00:FS00:Exception: Could not get an assignment

[Jan]

I dont see anything off in the logs right now. Out of curiosity: What PSU do you have?

[Petr_Z]

EVGA 1000GQ (1000W). I have fold without Problems for the month, but now I have the troubles with stability, also somehow there asre slow reactions of the System for example to the keybord during the folding
The Problems starts, when I have lost access to the advanced control (web control is still working). I guess the Problem is in the CPU, the crash happend when CPU starts the WU. The folding can increase the temperature to 95°C (Cinebench R20 does not have the chance to get the CPU over 72°C).

[Jan]

Hmm. Have you monitored your CPU temps prior? 95° is definitely in the critical area. Is there any OC going on? For stresstests you might also want to try Prime95, max power and heat production settings. FAH shouldnt be able to raise your temps above this test. Potentially, theres probably a range of other options regarding the true issue... but temps seem to be a start at those levels.

[Petr_Z]

Yes, there was OC which was working stable. There was no benchmark / stresstest which reveal the problem. But folding maybe did (after long term stress). I have switched back to auto and we will see, what happens. The Problem was in the CPU, it probably starts with throttling, what Looks like slow death of the System..
So, maybe the best way for Operation of Ryzen 3x ist simply do not overclock it at all, also Ryzen Master have some troubles.

[ipkh]

I would try changing the cpu slot to use less cores. If your system lags appreciably when folding, your CPU is having some contention issues.
Try setting the CPU to 20 cores. Leaving 2 real cores for the GPUs. If you are still having issues, try removing the Precision Boost Overdrive or limiting the OC it provides.

[PantherX]

Welcome to the F@H Forum Petr_Z,

If you're encountering screen lag while folding, it would be mainly due to GPU folding since the GPU has no scheduler/priority system. Here are links to disable hardware acceleration:
Windows: https://www.thewindowsclub.com/hardware ... -windows-7
Microsoft Office: https://www.solveyourtech.com/how-to-di ... ffice-365/
Firefox: https://support.mozilla.org/en-US/kb/ha ... inds-crash
Chrome: https://www.technize.net/google-chrome- ... eleration/
VLC: https://wiki.videolan.org/VLC_HowTo/Har ... eleration/

Alternatively, you can set the GPU Slot to fold only when idle. In Windows case, it is when the screensaver comes on. Thus, you can set a blank screensaver if you wanted: https://www.howtogeek.com/225305/how-to ... indows-10/

[Petr_Z]

Thank to all for the feedback. The failure was definitely in CPU (it was overclocked, and it Maybe degrades somehow slowly in the time). By Setting of the 22 used cores was the consumption too high (OC), temperatures were above near critical temperature of 95°C and the System starts to throttle.
Solution was Setting the amount of cores to 9. There is very low difference betwwen 22 and 9 cores.
The CPU behaves so, that if 9 cores are set, the CCD0 is completely utilized with full power, frequency is stable 4,1Ghz and temperature is ~90°C

The rest of 3 cores are utilized in CCD1, completely in by CCX0. There happens some Kind of power Control, because the frequency of the cores changes rellative wild between 2..4,1GHz). The temperature of CCD2 is in comparision to temperature of CCD1 low (70°C). So I continue now with the folding

So the Problem was the throttling of CCD1 (my understanding)

Folding at home is the best stres test for the whole System I know

[bruce]

Folding at home is the best stres test for the whole System I know

Sometimes I take that as complement (The developers wanted to use ALL of the unused resources) and sometimes I think of the number of times per day I've had to support Donors whose systems couldn't tolerate that much stress.

[uyaem]

Thank to all for the feedback. The failure was definitely in CPU (it was overclocked, and it Maybe degrades somehow slowly in the time). By Setting of the 22 used cores was the consumption too high (OC), temperatures were above near critical temperature of 95°C and the System starts to throttle.
Solution was Setting the amount of cores to 9. There is very low difference betwwen 22 and 9 cores.
The CPU behaves so, that if 9 cores are set, the CCD0 is completely utilized with full power, frequency is stable 4,1Ghz and temperature is ~90°C

The rest of 3 cores are utilized in CCD1, completely in by CCX0. There happens some Kind of power Control, because the frequency of the cores changes rellative wild between 2..4,1GHz). The temperature of CCD2 is in comparision to temperature of CCD1 low (70°C). So I continue now with the folding

So the Problem was the throttling of CCD1 (my understanding)

Folding at home is the best stres test for the whole System I know

Hi Petr,

I've had similar issues (no crash, but 95 celsius and LOUD fans), so I'm using Ryzen Master to limit CPU clocks to default speed (3800 for me). This allows me to go lower on Vcore (1.05V has had 0 crashes for me in 12 days, 24h/day), reducing power consumption by 40% with 21 cores, with only a small (~10%) loss on efficiency for folding (see https://foldingforum.org/viewtopic.php?f=38&t=34207).
Now sitting at a "luke warm" 68°C with the boxed cooler with relatively quiet fans.

[Petr_Z]

Hi Uyaem

I have the water cooling loop, so there was not the Problem with loudness. But thank you for your tip, it works (is usefull, I am also from Stuttgart end Energy effciciency is important). But the Problem was also, that the Ventilation of motherboard area with all components was switched off after my optimisation trials to reach silentest System as possible. Because there is no other vent in the component area, the RAM (with RGB) was also overheated because there was no forced air flow at all (natural thermal circulation was not enough)