Bug report, possibly DST related

[abe]

The following log seems to indicate a bug related to daylight savings time. Where I am we just switched from MST to MDT, so an extra hour was added to our clocks.

It looks like the software detected that the time was off by 1 hour, and tried to do something about it, but immediately afterwards it crashed. Fortunately it appears that it only lost small number of completed steps, from what I can tell.

Code: Select all

20:54:44:WU00:FS00:0xa7:Completed 100000 out of 2500000 steps (4%)
21:18:02:WU00:FS00:0xa7:Completed 125000 out of 2500000 steps (5%)
22:27:22:WARNING:WU00:FS00:Detected clock skew (1 hours 00 mins), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
22:27:22:WARNING:WU00:FS00:FahCore returned an unknown error code which probably indicates that it crashed
22:27:22:WARNING:WU00:FS00:FahCore returned: WU_STALLED (127 = 0x7f)
22:27:22:WU00:FS00:Starting
22:27:22:WARNING:WU00:FS00:AS lowered CPUs from 4 to 1
22:27:22:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\pc\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/v7/win/64bit/avx/Core_a7.fah/FahCore_a7.exe -dir 00 -suffix 01 -version 705 -lifeline 5256 -checkpoint 15 -np 1
22:27:23:WU00:FS00:Started FahCore on PID 4416
22:27:23:WU00:FS00:Core PID:212
22:27:23:WU00:FS00:FahCore 0xa7 started
22:27:23:WU00:FS00:0xa7:*********************** Log Started 2020-03-08T22:27:23Z ***********************
22:27:23:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
22:27:23:WU00:FS00:0xa7:       Type: 0xa7
22:27:23:WU00:FS00:0xa7:       Core: Gromacs
22:27:23:WU00:FS00:0xa7:       Args: -dir 00 -suffix 01 -version 705 -lifeline 4416 -checkpoint 15 -np 1
22:27:23:WU00:FS00:0xa7:************************************ CBang *************************************
22:27:23:WU00:FS00:0xa7:       Date: Oct 26 2019
22:27:23:WU00:FS00:0xa7:       Time: 01:38:25
22:27:23:WU00:FS00:0xa7:   Revision: c46a1a011a24143739ac7218c5a435f66777f62f
22:27:23:WU00:FS00:0xa7:     Branch: master
22:27:23:WU00:FS00:0xa7:   Compiler: Visual C++ 2008
22:27:23:WU00:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
22:27:23:WU00:FS00:0xa7:   Platform: win32 10
22:27:23:WU00:FS00:0xa7:       Bits: 64
22:27:23:WU00:FS00:0xa7:       Mode: Release
22:27:23:WU00:FS00:0xa7:************************************ System ************************************
22:27:23:WU00:FS00:0xa7:        CPU: Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz
22:27:23:WU00:FS00:0xa7:     CPU ID: GenuineIntel Family 6 Model 78 Stepping 3
22:27:23:WU00:FS00:0xa7:       CPUs: 4
22:27:23:WU00:FS00:0xa7:     Memory: 7.87GiB
22:27:23:WU00:FS00:0xa7:Free Memory: 4.49GiB
22:27:23:WU00:FS00:0xa7:    Threads: WINDOWS_THREADS
22:27:23:WU00:FS00:0xa7: OS Version: 6.2
22:27:23:WU00:FS00:0xa7:Has Battery: true
22:27:23:WU00:FS00:0xa7: On Battery: false
22:27:23:WU00:FS00:0xa7: UTC Offset: -6
22:27:23:WU00:FS00:0xa7:        PID: 212
22:27:23:WU00:FS00:0xa7:        CWD: C:\Users\pc\AppData\Roaming\FAHClient\work
22:27:23:WU00:FS00:0xa7:******************************** Build - libFAH ********************************
22:27:23:WU00:FS00:0xa7:    Version: 0.0.18
22:27:23:WU00:FS00:0xa7:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
22:27:23:WU00:FS00:0xa7:  Copyright: 2019 foldingathome.org
22:27:23:WU00:FS00:0xa7:   Homepage: https://foldingathome.org/
22:27:23:WU00:FS00:0xa7:       Date: Oct 26 2019
22:27:23:WU00:FS00:0xa7:       Time: 01:52:30
22:27:23:WU00:FS00:0xa7:   Revision: c1e3513b1bc0c16013668f2173ee969e5995b38e
22:27:23:WU00:FS00:0xa7:     Branch: master
22:27:23:WU00:FS00:0xa7:   Compiler: Visual C++ 2008
22:27:23:WU00:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
22:27:23:WU00:FS00:0xa7:   Platform: win32 10
22:27:23:WU00:FS00:0xa7:       Bits: 64
22:27:23:WU00:FS00:0xa7:       Mode: Release
22:27:23:WU00:FS00:0xa7:************************************ Build *************************************
22:27:23:WU00:FS00:0xa7:       SIMD: avx_256
22:27:23:WU00:FS00:0xa7:********************************************************************************
22:27:23:WU00:FS00:0xa7:Project: 14052 (Run 14, Clone 87, Gen 6)
22:27:23:WU00:FS00:0xa7:Unit: 0x000000070002894b5e0de9a6a15f6961
22:27:23:WU00:FS00:0xa7:Digital signatures verified
22:27:23:WU00:FS00:0xa7:Calling: mdrun -s frame6.tpr -o frame6.trr -cpi state.cpt -cpt 15 -nt 1
22:27:24:WU00:FS00:0xa7:Steps: first=15000000 total=2500000
22:27:24:WU00:FS00:0xa7:Completed 124981 out of 2500000 steps (4%)
22:27:26:WU00:FS00:0xa7:Completed 125000 out of 2500000 steps (5%)
22:51:00:WU00:FS00:0xa7:Completed 150000 out of 2500000 steps (6%)

[bruce]

FAH does detect "clock skew" and adjusts for it when the computer is off-line for an interval. This is the first time I've seen a report that it cause a crash or error 0x7f.

FAH's log follows the worldwide standard GMT which does not change when the local timezone changes to/from Daylight (aka Summer) time. My computer's local clock was adjusted last night at 2am-local but there's no indication in my log that anything changed ... and no clock skew message. I'm not sure what happened to yours. Was/is your computer's timezone correct? Is your OS configured to recognize that your local clock should be adjusted automatically for Daylight time?

[toTOW]

I usually see such warning messages after the system restarts from hibernation ... which should be avoided (or better, disabled) if you want to fold efficiently and within deadlines.

[abe]

Was/is your computer's timezone correct? Is your OS configured to recognize that your local clock should be adjusted automatically for Daylight time?

Yes, and it did switch automatically. I should clarify, I don't think that my OS switched my local clock at the time of the error, mine would have also changed at 2am or midnight, not sure exactly, but that would have been several hours before I even downloaded and installed the software. So it is possible that the 1 hour skew is not actually related to DST at all, I just thought it might be due to the coincidence.

I usually see such warning messages after the system restarts from hibernation ... which should be avoided (or better, disabled) if you want to fold efficiently and within deadlines.

All hibernation/sleep were already disabled on my OS. I am certain that this particular crash was not caused by it.

[foldy]

Seems like a bug in FAH but as it can easily resume from last checkpoint and DST only happens twice a year this is not a big issue.