Why does the log no longer echo further assignment retries?

If you're new to FAH and need help getting started or you have very basic questions, start here.

Moderators: Site Moderators, FAHC Science Team

Post Reply
BP2020
Posts: 46
Joined: Sun Apr 19, 2020 9:53 pm

Why does the log no longer echo further assignment retries?

Post by BP2020 »

I'm looking at the log on 7.6.20 I recently upgraded to from .13. I started the program around 20:18, I'm folding a WU for the GPU, I have no assigned unit for the CPU, then I see lines such as :

Code: Select all

20:19:04:WARNING:WU00:FS00:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
20:19:04:ERROR:WU00:FS00:Exception: Could not get an assignment.
Then around 20:21 again:

Code: Select all

20:21:40:WU00:FS00:Connecting to assign1.foldingathome.org:80
20:21:41:WARNING:WU00:FS00:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
20:21:41:WU00:FS00:Connecting to assign2.foldingathome.org:80
It can happen (but in my experience it's rare I have to wait for CPU WUs). But after that, I'm looking in the Advanced Control for the slot and I can see the retry counter going to 0 then restarting, the delay is now 14 mins, it's increasing etc. All of this is normal but I can't see any echo of the program trying to get a unit in the log after that, yet the retry counter has restarted many times. I did click refresh to make sure. It's been 3-4 retries now but no echo. The GPU WU steps are being echoed.

Is it only that the retries are not echoed or that the retry doesn't actually occur after the first two retries?

Full log (2 echoes, 9 attempts at this point, retry in 39 mins):

Code: Select all

*********************** Log Started 2020-10-22T20:18:47Z ***********************
20:18:47:******************************* libFAH ********************************
20:18:47:           Date: Oct 9 2020
20:18:47:           Time: 10:37:10
20:18:47:       Revision: 06b99f7701e0d3f883dd14a78b459ad27da23809
20:18:47:         Branch: master
20:18:47:       Compiler: Visual C++ 2015
20:18:47:        Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
20:18:47:       Platform: win32 10
20:18:47:           Bits: 32
20:18:47:           Mode: Release
20:18:47:****************************** FAHClient ******************************
20:18:47:        Version: 7.6.20
20:18:47:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
20:18:47:      Copyright: 2020 foldingathome.org
20:18:47:       Homepage: https://foldingathome.org/
20:18:47:           Date: Oct 12 2020
20:18:47:           Time: 15:02:06
20:18:47:       Revision: c858fe2a8342bfa3e116e00b394d8dfa322ecd18
20:18:47:         Branch: master
20:18:47:       Compiler: Visual C++ 2015
20:18:47:        Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
20:18:47:       Platform: win32 10
20:18:47:           Bits: 32
20:18:47:           Mode: Release
20:18:47:         Config: C:\Users\whatever\AppData\Roaming\FAHClient\config.xml
20:18:47:******************************** CBang ********************************
20:18:47:           Date: Oct 9 2020
20:18:47:           Time: 10:29:47
20:18:47:       Revision: ab0a6d9e35982b831a74cb2706c569fe46bac2af
20:18:47:         Branch: master
20:18:47:       Compiler: Visual C++ 2015
20:18:47:        Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
20:18:47:       Platform: win32 10
20:18:47:           Bits: 32
20:18:47:           Mode: Release
20:18:47:******************************* System ********************************
20:18:47:            CPU: Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
20:18:47:         CPU ID: GenuineIntel Family 6 Model 42 Stepping 7
20:18:47:           CPUs: 4
20:18:47:         Memory: 15.97GiB
20:18:47:    Free Memory: 8.79GiB
20:18:47:        Threads: WINDOWS_THREADS
20:18:47:     OS Version: 6.2
20:18:47:    Has Battery: false
20:18:47:     On Battery: false
20:18:47:     UTC Offset: -4
20:18:47:            PID: 6412
20:18:47:            CWD: C:\Users\whatever\AppData\Roaming\FAHClient
20:18:47:  Win32 Service: false
20:18:47:             OS: Windows 10 Enterprise
20:18:47:        OS Arch: AMD64
20:18:47:           GPUs: 1
20:18:47:          GPU 0: Bus:1 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
20:18:47:                 470/480/570/580/590]
20:18:47:           CUDA: Not detected: Failed to open dynamic library 'nvcuda.dll': The
20:18:47:                 specified module could not be found.
20:18:47:
20:18:47:OpenCL Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:3004.8
20:18:47:***********************************************************************
20:18:47:<config>
20:18:47:  <!-- Folding Slot Configuration -->
20:18:47:  <cause v='COVID_19'/>
20:18:47:
20:18:47:  <!-- Network -->
20:18:47:  <proxy v=':8080'/>
20:18:47:
20:18:47:  <!-- Slot Control -->
20:18:47:  <pause-on-start v='true'/>
20:18:47:  <power v='full'/>
20:18:47:
20:18:47:  <!-- User Information -->
20:18:47:  <passkey v='***'/>
20:18:47:  <team v=''/>
20:18:47:  <user v=''/>
20:18:47:
20:18:47:  <!-- Folding Slots -->
20:18:47:  <slot id='0' type='CPU'/>
20:18:47:  <slot id='1' type='GPU'>
20:18:47:    <pci-bus v='1'/>
20:18:47:    <pci-slot v='0'/>
20:18:47:  </slot>
20:18:47:</config>
20:18:47:Trying to access database...
20:18:47:Successfully acquired database lock
20:18:47:FS00:Initialized folding slot 00: cpu:3 - PAUSED by user
20:18:47:FS01:Initialized folding slot 01: gpu:1:0 Ellesmere XT [Radeon RX 470/480/570/580/590] - PAUSED by user
20:19:02:FS00:Unpaused
20:19:02:FS01:Unpaused
20:19:02:WU00:FS00:Connecting to assign1.foldingathome.org:80
20:19:02:WU01:FS01:Connecting to assign1.foldingathome.org:80
20:19:02:WARNING:WU00:FS00:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
20:19:02:WU00:FS00:Connecting to assign2.foldingathome.org:80
20:19:03:WARNING:WU00:FS00:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
20:19:03:WU00:FS00:Connecting to assign3.foldingathome.org:80
20:19:03:WU01:FS01:Assigned to work server 18.188.125.154
20:19:03:WU01:FS01:Requesting new work unit for slot 01: gpu:1:0 Ellesmere XT [Radeon RX 470/480/570/580/590] - READY from 18.188.125.154
20:19:03:WU01:FS01:Connecting to 18.188.125.154:8080
20:19:03:WARNING:WU00:FS00:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
20:19:03:WU00:FS00:Connecting to assign4.foldingathome.org:80
20:19:03:WARNING:WU00:FS00:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
20:19:03:ERROR:WU00:FS00:Exception: Could not get an assignment
20:19:03:WU00:FS00:Connecting to assign1.foldingathome.org:80
20:19:04:WARNING:WU00:FS00:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
20:19:04:WU00:FS00:Connecting to assign2.foldingathome.org:80
20:19:04:WARNING:WU00:FS00:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
20:19:04:WU00:FS00:Connecting to assign3.foldingathome.org:80
20:19:04:WU01:FS01:Downloading 12.00MiB
20:19:04:WARNING:WU00:FS00:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
20:19:04:WU00:FS00:Connecting to assign4.foldingathome.org:80
20:19:04:WARNING:WU00:FS00:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
20:19:04:ERROR:WU00:FS00:Exception: Could not get an assignment
20:19:06:WU01:FS01:Download complete
20:19:06:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:17308 run:0 clone:5096 gen:15 core:0x22 unit:0x0000001012bc7d9a5f8f11cd6b0ff4b5
20:19:06:WU01:FS01:Starting
20:19:06:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\whatever\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe -dir 01 -suffix 01 -version 706 -lifeline 6412 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -gpu-vendor amd -gpu 0 -gpu-usage 100
20:19:06:WU01:FS01:Started FahCore on PID 9624
20:19:07:WU01:FS01:Core PID:6788
20:19:07:WU01:FS01:FahCore 0x22 started
20:19:07:WU01:FS01:0x22:*********************** Log Started 2020-10-22T20:19:07Z ***********************
20:19:07:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
20:19:07:WU01:FS01:0x22:       Core: Core22
20:19:07:WU01:FS01:0x22:       Type: 0x22
20:19:07:WU01:FS01:0x22:    Version: 0.0.13
20:19:07:WU01:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
20:19:07:WU01:FS01:0x22:  Copyright: 2020 foldingathome.org
20:19:07:WU01:FS01:0x22:   Homepage: https://foldingathome.org/
20:19:07:WU01:FS01:0x22:       Date: Sep 19 2020
20:19:07:WU01:FS01:0x22:       Time: 02:35:58
20:19:07:WU01:FS01:0x22:   Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
20:19:07:WU01:FS01:0x22:     Branch: core22-0.0.13
20:19:07:WU01:FS01:0x22:   Compiler: Visual C++ 2015
20:19:07:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
20:19:07:WU01:FS01:0x22:             -DOPENMM_GIT_HASH="\"189320d0\""
20:19:07:WU01:FS01:0x22:   Platform: win32 10
20:19:07:WU01:FS01:0x22:       Bits: 64
20:19:07:WU01:FS01:0x22:       Mode: Release
20:19:07:WU01:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
20:19:07:WU01:FS01:0x22:             <peastman@stanford.edu>
20:19:07:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 706 -lifeline 9624 -checkpoint 15
20:19:07:WU01:FS01:0x22:             -opencl-platform 0 -opencl-device 0 -gpu-vendor amd -gpu 0
20:19:07:WU01:FS01:0x22:             -gpu-usage 100
20:19:07:WU01:FS01:0x22:************************************ libFAH ************************************
20:19:07:WU01:FS01:0x22:       Date: Sep 7 2020
20:19:07:WU01:FS01:0x22:       Time: 19:09:56
20:19:07:WU01:FS01:0x22:   Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
20:19:07:WU01:FS01:0x22:     Branch: HEAD
20:19:07:WU01:FS01:0x22:   Compiler: Visual C++ 2015
20:19:07:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
20:19:07:WU01:FS01:0x22:   Platform: win32 10
20:19:07:WU01:FS01:0x22:       Bits: 64
20:19:07:WU01:FS01:0x22:       Mode: Release
20:19:07:WU01:FS01:0x22:************************************ CBang *************************************
20:19:07:WU01:FS01:0x22:       Date: Sep 7 2020
20:19:07:WU01:FS01:0x22:       Time: 19:08:30
20:19:07:WU01:FS01:0x22:   Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
20:19:07:WU01:FS01:0x22:     Branch: HEAD
20:19:07:WU01:FS01:0x22:   Compiler: Visual C++ 2015
20:19:07:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
20:19:07:WU01:FS01:0x22:   Platform: win32 10
20:19:07:WU01:FS01:0x22:       Bits: 64
20:19:07:WU01:FS01:0x22:       Mode: Release
20:19:07:WU01:FS01:0x22:************************************ System ************************************
20:19:07:WU01:FS01:0x22:        CPU: Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
20:19:07:WU01:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 42 Stepping 7
20:19:07:WU01:FS01:0x22:       CPUs: 4
20:19:07:WU01:FS01:0x22:     Memory: 15.97GiB
20:19:07:WU01:FS01:0x22:Free Memory: 8.91GiB
20:19:07:WU01:FS01:0x22:    Threads: WINDOWS_THREADS
20:19:07:WU01:FS01:0x22: OS Version: 6.2
20:19:07:WU01:FS01:0x22:Has Battery: false
20:19:07:WU01:FS01:0x22: On Battery: false
20:19:07:WU01:FS01:0x22: UTC Offset: -4
20:19:07:WU01:FS01:0x22:        PID: 6788
20:19:07:WU01:FS01:0x22:        CWD: C:\Users\whatever\AppData\Roaming\FAHClient\work
20:19:07:WU01:FS01:0x22:************************************ OpenMM ************************************
20:19:07:WU01:FS01:0x22:   Revision: 189320d0
20:19:07:WU01:FS01:0x22:********************************************************************************
20:19:07:WU01:FS01:0x22:Project: 17308 (Run 0, Clone 5096, Gen 15)
20:19:07:WU01:FS01:0x22:Unit: 0x0000001012bc7d9a5f8f11cd6b0ff4b5
20:19:07:WU01:FS01:0x22:Reading tar file core.xml
20:19:07:WU01:FS01:0x22:Reading tar file integrator.xml.bz2
20:19:07:WU01:FS01:0x22:Reading tar file state.xml.bz2
20:19:07:WU01:FS01:0x22:Reading tar file system.xml.bz2
20:19:07:WU01:FS01:0x22:Digital signatures verified
20:19:07:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
20:19:07:WU01:FS01:0x22:Version 0.0.13
20:19:07:WU01:FS01:0x22:  Checkpoint write interval: 62500 steps (5%) [20 total]
20:19:07:WU01:FS01:0x22:  JSON viewer frame write interval: 12500 steps (1%) [100 total]
20:19:07:WU01:FS01:0x22:  XTC frame write interval: 125000 steps (10%) [10 total]
20:19:07:WU01:FS01:0x22:  Global context and integrator variables write interval: disabled
20:19:08:WU01:FS01:0x22:There are 3 platforms available.
20:19:08:WU01:FS01:0x22:Platform 0: Reference
20:19:08:WU01:FS01:0x22:Platform 1: CPU
20:19:08:WU01:FS01:0x22:Platform 2: OpenCL
20:19:08:WU01:FS01:0x22:  opencl-device 0 specified
20:19:13:FS00:Finishing
20:19:13:FS01:Finishing
20:19:20:WU01:FS01:0x22:Attempting to create OpenCL context:
20:19:20:WU01:FS01:0x22:  Configuring platform OpenCL
20:19:28:WU01:FS01:0x22:  Using OpenCL on platformId 0 and gpu 0
20:19:28:WU01:FS01:0x22:Completed 0 out of 1250000 steps (0%)
20:19:30:WU01:FS01:0x22:Checkpoint completed at step 0
20:20:03:WU00:FS00:Connecting to assign1.foldingathome.org:80
20:20:03:WARNING:WU00:FS00:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
20:20:03:WU00:FS00:Connecting to assign2.foldingathome.org:80
20:20:04:WARNING:WU00:FS00:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
20:20:04:WU00:FS00:Connecting to assign3.foldingathome.org:80
20:20:04:WARNING:WU00:FS00:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
20:20:04:WU00:FS00:Connecting to assign4.foldingathome.org:80
20:20:04:WARNING:WU00:FS00:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
20:20:04:ERROR:WU00:FS00:Exception: Could not get an assignment
20:21:40:WU00:FS00:Connecting to assign1.foldingathome.org:80
20:21:41:WARNING:WU00:FS00:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
20:21:41:WU00:FS00:Connecting to assign2.foldingathome.org:80
20:21:41:WU00:FS00:Assigned to work server 206.223.170.146
20:22:31:WU01:FS01:0x22:Completed 12500 out of 1250000 steps (1%)
20:25:32:WU01:FS01:0x22:Completed 25000 out of 1250000 steps (2%)
20:28:33:WU01:FS01:0x22:Completed 37500 out of 1250000 steps (3%)
20:31:34:WU01:FS01:0x22:Completed 50000 out of 1250000 steps (4%)
20:34:36:WU01:FS01:0x22:Completed 62500 out of 1250000 steps (5%)
20:34:37:WU01:FS01:0x22:Checkpoint completed at step 62500
20:37:38:WU01:FS01:0x22:Completed 75000 out of 1250000 steps (6%)
20:40:39:WU01:FS01:0x22:Completed 87500 out of 1250000 steps (7%)
20:43:40:WU01:FS01:0x22:Completed 100000 out of 1250000 steps (8%)
20:46:42:WU01:FS01:0x22:Completed 112500 out of 1250000 steps (9%)
20:49:44:WU01:FS01:0x22:Completed 125000 out of 1250000 steps (10%)
20:49:46:WU01:FS01:0x22:Checkpoint completed at step 125000
20:52:47:WU01:FS01:0x22:Completed 137500 out of 1250000 steps (11%)
20:55:49:WU01:FS01:0x22:Completed 150000 out of 1250000 steps (12%)
20:58:51:WU01:FS01:0x22:Completed 162500 out of 1250000 steps (13%)
21:01:53:WU01:FS01:0x22:Completed 175000 out of 1250000 steps (14%)
21:04:55:WU01:FS01:0x22:Completed 187500 out of 1250000 steps (15%)
21:04:57:WU01:FS01:0x22:Checkpoint completed at step 187500
21:07:59:WU01:FS01:0x22:Completed 200000 out of 1250000 steps (16%)
21:11:01:WU01:FS01:0x22:Completed 212500 out of 1250000 steps (17%)
21:14:02:WU01:FS01:0x22:Completed 225000 out of 1250000 steps (18%)
21:17:04:WU01:FS01:0x22:Completed 237500 out of 1250000 steps (19%)
21:20:05:WU01:FS01:0x22:Completed 250000 out of 1250000 steps (20%)
21:20:08:WU01:FS01:0x22:Checkpoint completed at step 250000
21:23:16:WU01:FS01:0x22:Completed 262500 out of 1250000 steps (21%)
21:26:18:WU01:FS01:0x22:Completed 275000 out of 1250000 steps (22%)
21:29:20:WU01:FS01:0x22:Completed 287500 out of 1250000 steps (23%)
21:32:21:WU01:FS01:0x22:Completed 300000 out of 1250000 steps (24%)
21:35:21:WU01:FS01:0x22:Completed 312500 out of 1250000 steps (25%)
21:35:23:WU01:FS01:0x22:Checkpoint completed at step 312500
21:38:24:WU01:FS01:0x22:Completed 325000 out of 1250000 steps (26%)
Image
Intel Core i5-2500K CPU @ 3.30GHz @ 98%, 4 cores
Asus ROG STRIX RX480 8Gb
Top
PantherX
Site Moderator
Posts: 7020
Joined: Wed Dec 23, 2009 9:33 am
Hardware configuration: V7.6.21 -> Multi-purpose 24/7
Windows 10 64-bit
CPU:2/3/4/6 -> Intel i7-6700K
GPU:1 -> Nvidia GTX 1080 Ti
§
Retired:
2x Nvidia GTX 1070
Nvidia GTX 675M
Nvidia GTX 660 Ti
Nvidia GTX 650 SC
Nvidia GTX 260 896 MB SOC
Nvidia 9600GT 1 GB OC
Nvidia 9500M GS
Nvidia 8800GTS 320 MB

Intel Core i7-860
Intel Core i7-3840QM
Intel i3-3240
Intel Core 2 Duo E8200
Intel Core 2 Duo E6550
Intel Core 2 Duo T8300
Intel Pentium E5500
Intel Pentium E5400
Location: Land Of The Long White Cloud
Contact:
Contact PantherX

Re: Why does the log no longer echo further assignment retri

Post by PantherX »

Under normal conditions, it would always show retries regardless of the attempts. In your case, I believe that once your client got to this:
20:21:41:WU00:FS00:Assigned to work server 206.223.170.146

It might have encountered a network issue (https://github.com/FoldingAtHome/fah-issues/issues/983) and the only way to recover is by a full restart of the client. BTW, you may want to update to the latest version which is 7.6.21 since there's a bug in 7.6.20 on Windows when it comes to auto-startup.
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time

Welcome To The F@H Support Forum Ӂ Troubleshooting Bad WUs Ӂ Troubleshooting Server Connectivity Issues
Top
BP2020
Posts: 46
Joined: Sun Apr 19, 2020 9:53 pm

Re: Why does the log no longer echo further assignment retri

Post by BP2020 »

Thank you, I'm pretty sure you're right (network issue and something failed to recover so the slot is dead until a restart), the GPU WU unit completed without me getting a CPU one, and when that was done I restarted the application and I got a CPU WU. Next time that happens I'll just wait for a checkpoint then pause, and exit/restart. It was just a tiny wasted opportunity for me to contribute modestly during that time but since the PC is not unattended and I check things out regularly, including the log, it won't be an issue for me anymore.

Also thanks for the heads up about 7.6.21.
Image
Intel Core i5-2500K CPU @ 3.30GHz @ 98%, 4 cores
Asus ROG STRIX RX480 8Gb
Top
Post Reply

Return to “New Donors start here”