WU status resets every 20-30 mins

Moderators: Site Moderators, FAHC Science Team

Post Reply
TarMoh
Posts: 13
Joined: Sun Mar 22, 2020 4:40 pm
Hardware configuration: Intel Core i5-8400
Radeon RX 590
16GB 2400MHz RAM C16

WU status resets every 20-30 mins

Post by TarMoh »

For some reason, whenever I’m running the GPU folding, its status resents every now and then and goes back to 0 - and this is when it was just starting on it. One time, it was just 12% completed, 5% another, and 15% another. How can I diagnose the issue? BTW I’m running an RX 590 with an i5-8400. The Project number is 11761.
Joe_H
Site Admin
Posts: 7870
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: WU status resets every 20-30 mins

Post by Joe_H »

Could you post your log file? Something could show up there that would give us an idea what might be going wrong.

Directions on locating and posting your log are part of this Welcome topic - viewtopic.php?f=50&t=26036
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
TarMoh
Posts: 13
Joined: Sun Mar 22, 2020 4:40 pm
Hardware configuration: Intel Core i5-8400
Radeon RX 590
16GB 2400MHz RAM C16

Re: WU status resets every 20-30 mins

Post by TarMoh »

Code: Select all

02:25:35:WU01:FS01:Starting
02:25:35:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\green\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/v7/win/64bit/Core_22.fah/FahCore_22.exe -dir 01 -suffix 01 -version 705 -lifeline 12988 -checkpoint 15 -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
02:25:35:WU01:FS01:Started FahCore on PID 19156
02:25:35:WU01:FS01:Core PID:21292
02:25:35:WU01:FS01:FahCore 0x22 started
02:25:35:WU01:FS01:0x22:*********************** Log Started 2020-03-30T02:25:35Z ***********************
02:25:35:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
02:25:35:WU01:FS01:0x22:       Type: 0x22
02:25:35:WU01:FS01:0x22:       Core: Core22
02:25:35:WU01:FS01:0x22:    Website: https://foldingathome.org/
02:25:35:WU01:FS01:0x22:  Copyright: (c) 2009-2018 foldingathome.org
02:25:35:WU01:FS01:0x22:     Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
02:25:35:WU01:FS01:0x22:             <rafal.wiewiora@choderalab.org>
02:25:35:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 705 -lifeline 19156 -checkpoint 15
02:25:35:WU01:FS01:0x22:             -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
02:25:35:WU01:FS01:0x22:     Config: <none>
02:25:35:WU01:FS01:0x22:************************************ Build *************************************
02:25:35:WU01:FS01:0x22:    Version: 0.0.2
02:25:35:WU01:FS01:0x22:       Date: Dec 6 2019
02:25:35:WU01:FS01:0x22:       Time: 21:30:31
02:25:35:WU01:FS01:0x22: Repository: Git
02:25:35:WU01:FS01:0x22:   Revision: abeb39247cc72df5af0f63723edafadb23d5dfbe
02:25:35:WU01:FS01:0x22:     Branch: HEAD
02:25:35:WU01:FS01:0x22:   Compiler: Visual C++ 2008
02:25:35:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
02:25:35:WU01:FS01:0x22:   Platform: win32 10
02:25:35:WU01:FS01:0x22:       Bits: 64
02:25:35:WU01:FS01:0x22:       Mode: Release
02:25:35:WU01:FS01:0x22:************************************ System ************************************
02:25:35:WU01:FS01:0x22:        CPU: Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz
02:25:35:WU01:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 158 Stepping 10
02:25:35:WU01:FS01:0x22:       CPUs: 6
02:25:35:WU01:FS01:0x22:     Memory: 15.94GiB
02:25:35:WU01:FS01:0x22:Free Memory: 8.38GiB
02:25:35:WU01:FS01:0x22:    Threads: WINDOWS_THREADS
02:25:35:WU01:FS01:0x22: OS Version: 6.2
02:25:35:WU01:FS01:0x22:Has Battery: false
02:25:35:WU01:FS01:0x22: On Battery: false
02:25:35:WU01:FS01:0x22: UTC Offset: -4
02:25:35:WU01:FS01:0x22:        PID: 21292
02:25:35:WU01:FS01:0x22:        CWD: C:\Users\green\AppData\Roaming\FAHClient\work
02:25:35:WU01:FS01:0x22:         OS: Windows 10 Home
02:25:35:WU01:FS01:0x22:    OS Arch: AMD64
02:25:35:WU01:FS01:0x22:********************************************************************************
02:25:35:WU01:FS01:0x22:Project: 11742 (Run 0, Clone 4836, Gen 20)
02:25:35:WU01:FS01:0x22:Unit: 0x000000218ca304f15e69cdc172d101af
02:25:35:WU01:FS01:0x22:Digital signatures verified
02:25:35:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
02:25:35:WU01:FS01:0x22:Version 0.0.2
02:25:46:WU01:FS01:0x22:Completed 0 out of 2000000 steps (0%)
02:25:46:WU01:FS01:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
02:27:07:WU02:FS00:0xa7:Completed 152500 out of 250000 steps (61%)
02:28:38:WU02:FS00:0xa7:Completed 155000 out of 250000 steps (62%)
02:30:09:WU02:FS00:0xa7:Completed 157500 out of 250000 steps (63%)
02:31:41:WU02:FS00:0xa7:Completed 160000 out of 250000 steps (64%)
02:33:12:WU02:FS00:0xa7:Completed 162500 out of 250000 steps (65%)
02:34:44:WU02:FS00:0xa7:Completed 165000 out of 250000 steps (66%)
02:36:14:WU02:FS00:0xa7:Completed 167500 out of 250000 steps (67%)
02:37:05:WARNING:WU01:FS01:FahCore returned an unknown error code which probably indicates that it crashed
02:37:05:WARNING:WU01:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
02:37:05:WU01:FS01:Starting
02:37:05:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\green\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/v7/win/64bit/Core_22.fah/FahCore_22.exe -dir 01 -suffix 01 -version 705 -lifeline 12988 -checkpoint 15 -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
02:37:05:WU01:FS01:Started FahCore on PID 18668
02:37:05:WU01:FS01:Core PID:21212
02:37:05:WU01:FS01:FahCore 0x22 started
02:37:06:WU01:FS01:0x22:*********************** Log Started 2020-03-30T02:37:05Z ***********************
02:37:06:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
02:37:06:WU01:FS01:0x22:       Type: 0x22
02:37:06:WU01:FS01:0x22:       Core: Core22
02:37:06:WU01:FS01:0x22:    Website: https://foldingathome.org/
02:37:06:WU01:FS01:0x22:  Copyright: (c) 2009-2018 foldingathome.org
02:37:06:WU01:FS01:0x22:     Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
02:37:06:WU01:FS01:0x22:             <rafal.wiewiora@choderalab.org>
02:37:06:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 705 -lifeline 18668 -checkpoint 15
02:37:06:WU01:FS01:0x22:             -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
02:37:06:WU01:FS01:0x22:     Config: <none>
02:37:06:WU01:FS01:0x22:************************************ Build *************************************
02:37:06:WU01:FS01:0x22:    Version: 0.0.2
02:37:06:WU01:FS01:0x22:       Date: Dec 6 2019
02:37:06:WU01:FS01:0x22:       Time: 21:30:31
02:37:06:WU01:FS01:0x22: Repository: Git
02:37:06:WU01:FS01:0x22:   Revision: abeb39247cc72df5af0f63723edafadb23d5dfbe
02:37:06:WU01:FS01:0x22:     Branch: HEAD
02:37:06:WU01:FS01:0x22:   Compiler: Visual C++ 2008
02:37:06:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
02:37:06:WU01:FS01:0x22:   Platform: win32 10
02:37:06:WU01:FS01:0x22:       Bits: 64
02:37:06:WU01:FS01:0x22:       Mode: Release
02:37:06:WU01:FS01:0x22:************************************ System ************************************
02:37:06:WU01:FS01:0x22:        CPU: Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz
02:37:06:WU01:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 158 Stepping 10
02:37:06:WU01:FS01:0x22:       CPUs: 6
02:37:06:WU01:FS01:0x22:     Memory: 15.94GiB
02:37:06:WU01:FS01:0x22:Free Memory: 8.43GiB
02:37:06:WU01:FS01:0x22:    Threads: WINDOWS_THREADS
02:37:06:WU01:FS01:0x22: OS Version: 6.2
02:37:06:WU01:FS01:0x22:Has Battery: false
02:37:06:WU01:FS01:0x22: On Battery: false
02:37:06:WU01:FS01:0x22: UTC Offset: -4
02:37:06:WU01:FS01:0x22:        PID: 21212
02:37:06:WU01:FS01:0x22:        CWD: C:\Users\green\AppData\Roaming\FAHClient\work
02:37:06:WU01:FS01:0x22:         OS: Windows 10 Home
02:37:06:WU01:FS01:0x22:    OS Arch: AMD64
02:37:06:WU01:FS01:0x22:********************************************************************************
02:37:06:WU01:FS01:0x22:Project: 11742 (Run 0, Clone 4836, Gen 20)
02:37:06:WU01:FS01:0x22:Unit: 0x000000218ca304f15e69cdc172d101af
02:37:06:WU01:FS01:0x22:Digital signatures verified
02:37:06:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
02:37:06:WU01:FS01:0x22:Version 0.0.2
02:37:16:WU01:FS01:0x22:Completed 0 out of 2000000 steps (0%)
02:37:16:WU01:FS01:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
02:37:52:WU02:FS00:0xa7:Completed 170000 out of 250000 steps (68%)
02:39:27:WU02:FS00:0xa7:Completed 172500 out of 250000 steps (69%)
02:40:58:WU02:FS00:0xa7:Completed 175000 out of 250000 steps (70%)
02:42:29:WU02:FS00:0xa7:Completed 177500 out of 250000 steps (71%)
02:44:00:WU02:FS00:0xa7:Completed 180000 out of 250000 steps (72%)
02:45:31:WU02:FS00:0xa7:Completed 182500 out of 250000 steps (73%)
02:47:04:WU02:FS00:0xa7:Completed 185000 out of 250000 steps (74%)
02:48:17:WARNING:WU01:FS01:FahCore returned an unknown error code which probably indicates that it crashed
02:48:17:WARNING:WU01:FS01:FahCore returned: WU_STALLED (127 = 0x7f)
02:48:17:WU01:FS01:Starting
02:48:17:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\green\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/v7/win/64bit/Core_22.fah/FahCore_22.exe -dir 01 -suffix 01 -version 705 -lifeline 12988 -checkpoint 15 -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
02:48:18:WU01:FS01:Started FahCore on PID 14620
02:48:18:WU01:FS01:Core PID:18808
02:48:18:WU01:FS01:FahCore 0x22 started
02:48:18:WU01:FS01:0x22:*********************** Log Started 2020-03-30T02:48:18Z ***********************
02:48:18:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
02:48:18:WU01:FS01:0x22:       Type: 0x22
02:48:18:WU01:FS01:0x22:       Core: Core22
02:48:18:WU01:FS01:0x22:    Website: https://foldingathome.org/
02:48:18:WU01:FS01:0x22:  Copyright: (c) 2009-2018 foldingathome.org
02:48:18:WU01:FS01:0x22:     Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
02:48:18:WU01:FS01:0x22:             <rafal.wiewiora@choderalab.org>
02:48:18:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 705 -lifeline 14620 -checkpoint 15
02:48:18:WU01:FS01:0x22:             -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
02:48:18:WU01:FS01:0x22:     Config: <none>
02:48:18:WU01:FS01:0x22:************************************ Build *************************************
02:48:18:WU01:FS01:0x22:    Version: 0.0.2
02:48:18:WU01:FS01:0x22:       Date: Dec 6 2019
02:48:18:WU01:FS01:0x22:       Time: 21:30:31
02:48:18:WU01:FS01:0x22: Repository: Git
02:48:18:WU01:FS01:0x22:   Revision: abeb39247cc72df5af0f63723edafadb23d5dfbe
02:48:18:WU01:FS01:0x22:     Branch: HEAD
02:48:18:WU01:FS01:0x22:   Compiler: Visual C++ 2008
02:48:18:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
02:48:18:WU01:FS01:0x22:   Platform: win32 10
02:48:18:WU01:FS01:0x22:       Bits: 64
02:48:18:WU01:FS01:0x22:       Mode: Release
02:48:18:WU01:FS01:0x22:************************************ System ************************************
02:48:18:WU01:FS01:0x22:        CPU: Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz
02:48:18:WU01:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 158 Stepping 10
02:48:18:WU01:FS01:0x22:       CPUs: 6
02:48:18:WU01:FS01:0x22:     Memory: 15.94GiB
02:48:18:WU01:FS01:0x22:Free Memory: 8.42GiB
02:48:18:WU01:FS01:0x22:    Threads: WINDOWS_THREADS
02:48:18:WU01:FS01:0x22: OS Version: 6.2
02:48:18:WU01:FS01:0x22:Has Battery: false
02:48:18:WU01:FS01:0x22: On Battery: false
02:48:18:WU01:FS01:0x22: UTC Offset: -4
02:48:18:WU01:FS01:0x22:        PID: 18808
02:48:18:WU01:FS01:0x22:        CWD: C:\Users\green\AppData\Roaming\FAHClient\work
02:48:18:WU01:FS01:0x22:         OS: Windows 10 Home
02:48:18:WU01:FS01:0x22:    OS Arch: AMD64
02:48:18:WU01:FS01:0x22:********************************************************************************
02:48:18:WU01:FS01:0x22:Project: 11742 (Run 0, Clone 4836, Gen 20)
02:48:18:WU01:FS01:0x22:Unit: 0x000000218ca304f15e69cdc172d101af
02:48:18:WU01:FS01:0x22:Digital signatures verified
02:48:18:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
02:48:18:WU01:FS01:0x22:Version 0.0.2
02:48:29:WU01:FS01:0x22:Completed 0 out of 2000000 steps (0%)
02:48:29:WU01:FS01:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
02:48:38:WU02:FS00:0xa7:Completed 187500 out of 250000 steps (75%)
02:49:17:FS01:Paused
02:49:17:FS01:Shutting core down
02:49:17:WU01:FS01:0x22:WARNING:Console control signal 1 on PID 18808
02:49:17:WU01:FS01:0x22:Exiting, please wait. . .
02:49:17:WU01:FS01:0x22:Folding@home Core Shutdown: INTERRUPTED
02:49:17:WU01:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
02:50:14:Removing old file 'configs/config-20200326-131957.xml'
02:50:14:Saving configuration to config.xml
02:50:14:<config>
02:50:14:  <!-- Network -->
02:50:14:  <proxy v=':8080'/>
02:50:14:
02:50:14:  <!-- Slot Control -->
02:50:14:  <power v='full'/>
02:50:14:
02:50:14:  <!-- User Information -->
02:50:14:  <passkey v='********************************'/>
02:50:14:  <team v='223518'/>
02:50:14:  <user v='Tarran_M'/>
02:50:14:
02:50:14:  <!-- Folding Slots -->
02:50:14:  <slot id='0' type='CPU'/>
02:50:14:  <slot id='1' type='GPU'>
02:50:14:    <paused v='true'/>
02:50:14:  </slot>
02:50:14:</config>
02:50:19:WU02:FS00:0xa7:Completed 190000 out of 250000 steps (76%)
02:51:55:WU02:FS00:0xa7:Completed 192500 out of 250000 steps (77%)
Joe_H
Site Admin
Posts: 7870
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: WU status resets every 20-30 mins

Post by Joe_H »

One possibility is a WU that is borderline stable. However if it gets too many errors, then it will be stopped and returned as faulty.

Does this happen on every GPU WU you get? And is your GPU overclocked? If overclocked, try reducing that by a bit and see if the errors stop.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
TarMoh
Posts: 13
Joined: Sun Mar 22, 2020 4:40 pm
Hardware configuration: Intel Core i5-8400
Radeon RX 590
16GB 2400MHz RAM C16

Re: WU status resets every 20-30 mins

Post by TarMoh »

Yeah, it happens to every WU. For the past week for so, I've not been able to complete a GPU WU assignment - it keeps on resetting. My GPU is not overclocked, but I can try a slight underclock in Afterburner, maybe that could help.

Update: I did a test with Afterburner open, and it was at a stable 1580 MHz (2000MHz Mem Clock, both of these are stock) and leveled off at 73C
Joe_H
Site Admin
Posts: 7870
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: WU status resets every 20-30 mins

Post by Joe_H »

Are those stock clocks for the reference version or are they factory overclock numbers? A few have found they need to reduce to reference clocks or slightly above, the factory overclock speeds were not stable for GPU folding.

Only other thing I can think of at the moment is that folding is not too dependent on the RAM speed, so clock on that can be reduced a bit without much effect on the folding speed. That can give you a bit more thermal headroom for other components on the GPU card.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
TarMoh
Posts: 13
Joined: Sun Mar 22, 2020 4:40 pm
Hardware configuration: Intel Core i5-8400
Radeon RX 590
16GB 2400MHz RAM C16

Re: WU status resets every 20-30 mins

Post by TarMoh »

You may be correct, the card I am using is factory overclocked. What would be an appropriate speed for this? 1500MHz, or even lower? As for RAM, I'm only using 2400MHz, so I don't think that's an issue
toTOW
Site Moderator
Posts: 6309
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:

Re: WU status resets every 20-30 mins

Post by toTOW »

I'm pretty sure you're not using a v6 client. Thread moved to the appropriate forum : V7.5.1 Public Release Windows/Linux/MacOS X
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
TarMoh
Posts: 13
Joined: Sun Mar 22, 2020 4:40 pm
Hardware configuration: Intel Core i5-8400
Radeon RX 590
16GB 2400MHz RAM C16

Re: WU status resets every 20-30 mins

Post by TarMoh »

Whoops, I wasn't aware that this was specific to V6. I am running V7.5.1
Joe_H
Site Admin
Posts: 7870
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: WU status resets every 20-30 mins

Post by Joe_H »

TarMoh wrote:You may be correct, the card I am using is factory overclocked. What would be an appropriate speed for this? 1500MHz, or even lower? As for RAM, I'm only using 2400MHz, so I don't think that's an issue
Standard clock speed I found on one list is 1469 w/ boost to 1545. Give it a try at 1500 and see if that is enough.

Another approach some have used with Afterburner is to put in power and/or temperature limit and let the card manage its clock to stay within those.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
TarMoh
Posts: 13
Joined: Sun Mar 22, 2020 4:40 pm
Hardware configuration: Intel Core i5-8400
Radeon RX 590
16GB 2400MHz RAM C16

Re: WU status resets every 20-30 mins

Post by TarMoh »

I’ve been running it for a few hours now, and it seems to be working fine now. Thanks for the help!
Post Reply