FahCore21.exe Crashing After Upload

[ryoungblood]

I appear to be having a recurring issue with Core 21 crashing immediately after upload. I am currently using 378.92 drivers.

FAHClient Log:

17:37:57:WU02:FS01:0x18:Completed 5000000 out of 5000000 steps (100%)
17:37:58:WU01:FS01:Connecting to 171.67.108.45:80
17:37:58:WU01:FS01:Assigned to work server 140.163.4.245
17:37:58:WU01:FS01:Requesting new work unit for slot 01: RUNNING gpu:1:GP102 [GeForce GTX 1080 Ti] from 140.163.4.245
17:37:58:WU01:FS01:Connecting to 140.163.4.245:8080
17:37:59:WU01:FS01:Downloading 14.49MiB
17:38:01:WU02:FS01:0x18:Saving result file logfile_01.txt
17:38:01:WU02:FS01:0x18:Saving result file checkpointState.xml
17:38:02:WU01:FS01:Download complete
17:38:02:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:10496 run:15 clone:20 gen:39 core:0x21 unit:0x000000338ca304f558895a6e9c30172b
17:38:02:WU02:FS01:0x18:Saving result file checkpt.crc
17:38:02:WU02:FS01:0x18:Saving result file log.txt
17:38:02:WU02:FS01:0x18:Saving result file positions.xtc
17:38:03:WU02:FS01:0x18:Folding@home Core Shutdown: FINISHED_UNIT
17:38:03:WU02:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
17:38:03:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:10490 run:382 clone:0 gen:736 core:0x18 unit:0x000003698ca304f45537e916316ca4ff
17:38:03:WU02:FS01:Uploading 6.64MiB to 140.163.4.244
17:38:03:WU01:FS01:Starting
17:38:03:WU02:FS01:Connecting to 140.163.4.244:8080
17:38:03:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Command Center/AppData/Roaming/FAHClient/cores/fahwebx.stanford.edu/cores/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe" -dir 01 -suffix 01 -version 704 -lifeline 6988 -checkpoint 30 -gpu 0 -gpu-vendor nvidia
17:38:03:WU01:FS01:Started FahCore on PID 7184
17:38:03:WU01:FS01:Core PID:5944
17:38:03:WU01:FS01:FahCore 0x21 started
17:38:05:WU01:FS01:0x21:*********************** Log Started 2017-05-06T17:38:05Z ***********************
17:38:05:WU01:FS01:0x21:Project: 10496 (Run 15, Clone 20, Gen 39)
17:38:05:WU01:FS01:0x21:Unit: 0x000000338ca304f558895a6e9c30172b
17:38:05:WU01:FS01:0x21:CPU: 0x00000000000000000000000000000000
17:38:05:WU01:FS01:0x21:Machine: 1
17:38:05:WU01:FS01:0x21:Reading tar file core.xml
17:38:05:WU01:FS01:0x21:Reading tar file system.xml
17:38:06:WU01:FS01:0x21:Reading tar file integrator.xml
17:38:06:WU01:FS01:0x21:Reading tar file state.xml
17:38:08:WU01:FS01:0x21:Digital signatures verified
17:38:08:WU01:FS01:0x21:Folding@home GPU Core21 Folding@home Core
17:38:08:WU01:FS01:0x21:Version 0.0.18
17:38:09:WU02:FS01:Upload 93.12%
17:38:13:WU02:FS01:Upload complete
17:38:13:WU02:FS01:Server responded WORK_ACK (400)
17:38:13:WU02:FS01:Final credit estimate, 93399.00 points
17:38:13:WU02:FS01:Cleaning up
18:28:38:WARNING:WU01:FS01:FahCore returned: FAILED_1 (0 = 0x0)


Windows Event Log

Faulting application name: FahCore_21.exe, version: 0.0.0.0, time stamp: 0x588257cc
Faulting module name: ntdll.dll, version: 10.0.14393.479, time stamp: 0x5825887f
Exception code: 0xc0000374
Fault offset: 0x00000000000f8283
Faulting process id: 0x3234
Faulting application start time: 0x01d2b97bd895c739
Faulting application path: C:\Users\Command Center\AppData\Roaming\FAHClient\cores\fahwebx.stanford.edu\cores\Win32\AMD64\NVIDIA\Fermi\Core_21.fah\FahCore_21.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report Id: 44e303f8-6a28-41f9-87fe-b213dccee1a8
Faulting package full name:
Faulting package-relative application ID:


It appears that the WU uploads then the core crashes. Windows makes me acknowledge the crashed program via a text box before it reports as failed within the FAH Client.

System Specs

[bruce]

The failure has nothing to do with the upload. WU02 finished an uploaded successfully and "Cleaning up" seems to have worked correctly. WU01 was downloaded and FahCore_21 was started and then failed. I really doubt there was any interaction between the processing of WU02 and WU01.

Did the client restart WU01 successfully or was it dumped? (after the message 18:28:38:WARNING:WU01:FS01:FahCore returned: FAILED_1 (0 = 0x0))

The Windows exception code 0xc0000374 indicates a heap corruption - to determine the solution, you would need to debug the crash in a debugger to figure out who is corrupting the heap. Heap corruption can also occur spontaneously on overclocked systems.

[rwh202]

I wonder if it's something to do with this being a very quick system - the core finishes and starts a new WU in less than a second. I normally have a delay of a few seconds. Only a guess, but maybe something isn't being cleared out fully before the core is fired up again? Anti virus still scanning the fresh WU?

[Nert]

I've encountered this problem as well in the past while running Windows. I would have been running a different version of the video drivers at the time.

Here is the thread that I started: viewtopic.php?f=19&t=28709

Speculation at the time was hardware problem. I believe that speculation was wrong.

All of the symptoms I noticed are described by the original poster:

1) Previous core finishes correctly
2) New core crashes on startup following upload/download.
3) Windows indicates heap corruption problem.

RWH's speculation about a timing related bug makes sense to me ... there is some bug related to one core finishing and a new core starting up that causes heap corruption. It's intermittent and timing related. There are now two instances of this happening. I'm just spit balling here, but thought I'd share my recollection that this particular problem has occurred previously.

[bruce]

I can't think of a relationship between closing one core and cleaning up the files of one WU and starting another copy of the (same or different) core on another WU that would corrupt the heap. Those events happen regularly for people who run more than one GPU though rarely at "exactly" the same time.

(Just because I think there's no relationship doesn't prove anything. If I happen to be wrong, somebody is going to have to figure out how to gather enough information for development to identify the problem. In the passage I quoted above, it says "you would need to debug the crash in a debugger to figure out who is corrupting the heap.")

The fact that Windows provides a popup that must be responded to before continuing means that it has temporarily preserved a lot of useful information that could be of help but somebody would need to produce a mini-dump of that task BEFORE closing that popup. Can somebody do that for us?

[ryoungblood]

Nothing restarted after it crashed.

I'm not incredibly technical but next time it comes up I will go with the option that says debug. I'll play around with Visual Studio and see if I can get anything useful to post here.

It happens once every few days, so I should be expecting another one pretty soon. CPU isn't OC'ed, the 3 GPU (1080 Ti/1080 Ti/1070) in the system have a slight OC (+80/+80).

[jcoffland]

The second WU crashing during the first WUs clean up may just be a coincidence. However, it is possible that the core is accessing the GPU driver during clean up and that is causing the crash. This would be easy to test. Run a core 0x21 on a WU as normal. Then see if you can crash it simply by running another instance of the core manually. You could run it like this:

Code: Select all

C:\Users\Command Center\AppData\Roaming\FAHClient\cores\fahwebx.stanford.edu\cores\Win32\AMD64\NVIDIA\Fermi\Core_21.fah\FahCore_21.exe --info

[ryoungblood]

@ryoungblood: If you use visual studio debug the most valueable part would be the callstack of the exception and create a dump file.

Good to know. Nothing has happened yet.

[ryoungblood]

This probably isn't much help.

I couldn't get the call stack on that past error.

https://i.imgur.com/LnIQQKq.png

[foldy]

Got this error also once, upload succeeded but the fahcore_21 error popup occured. When I click close then a new FahCore_21 is started successfully.

Only error message in log is
17:02:14:WARNING:WU02:FS01:FahCore returned: FAILED_3 (255 = 0xff)

Code: Select all

*********************** Log Started 2017-12-15T16:19:44Z ***********************
16:19:44:************************* Folding@home Client *************************
16:19:44:        Website: http://folding.stanford.edu/
16:19:44:      Copyright: (c) 2009-2016 Stanford University
16:19:44:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
16:19:44:           Args: 
16:19:44:         Config: C:\ProgramData\FAHClient\config.xml
16:19:44:******************************** Build ********************************
16:19:44:        Version: 7.4.16
16:19:44:           Date: Jan 6 2017
16:19:44:           Time: 00:25:14
16:19:44:     Repository: Git
16:19:44:       Revision: a9e9e27dc2ee6ff01398c439677bc27f6cb74032
16:19:44:         Branch: master
16:19:44:       Compiler: Visual C++ 2008
16:19:44:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox -arch:SSE /MT
16:19:44:       Platform: win32 10
16:19:44:           Bits: 32
16:19:44:           Mode: Release
16:19:44:******************************* System ********************************
16:19:44:            CPU: Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
16:19:44:         CPU ID: GenuineIntel Family 6 Model 42 Stepping 7
16:19:44:           CPUs: 4
16:19:44:         Memory: 15.60GiB
16:19:44:    Free Memory: 13.33GiB
16:19:44:        Threads: WINDOWS_THREADS
16:19:44:     OS Version: 6.1
16:19:44:    Has Battery: false
16:19:44:     On Battery: false
16:19:44:     UTC Offset: 1
16:19:44:            PID: 3828
16:19:44:            CWD: C:\ProgramData\FAHClient
16:19:44:             OS: Windows 7 Professional
16:19:44:        OS Arch: AMD64
16:19:44:           GPUs: 2
16:19:44:          GPU 0: Bus:1 Slot:0 Func:0 NVIDIA:7 GP102 [GeForce GTX 1080 Ti] 11380
16:19:44:          GPU 1: Bus:2 Slot:0 Func:0 NVIDIA:7 GP102 [GeForce GTX 1080 Ti] 11380
16:19:44:  CUDA Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:6.1 Driver:9.1
16:19:44:  CUDA Device 1: Platform:0 Device:1 Bus:2 Slot:0 Compute:6.1 Driver:9.1
16:19:44:OpenCL Device 2: Platform:2 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:388.59
16:19:44:OpenCL Device 3: Platform:2 Device:1 Bus:2 Slot:0 Compute:1.2 Driver:388.59
16:19:44:  Win32 Service: false
16:19:44:***********************************************************************
16:19:44:<config>
16:19:44:  <!-- Client Control -->
16:19:44:  <disable-sleep-when-active v='false'/>
16:19:44:
16:19:44:  <!-- Folding Core -->
16:19:44:  <checkpoint v='5'/>
16:19:44:  <core-priority v='low'/>
16:19:44:  <gpu-usage v='10'/>
16:19:44:
16:19:44:  <!-- Folding Slot Configuration -->
16:19:44:  <client-type v='beta'/>
16:19:44:  <max-packet-size v='big'/>
16:19:44:
16:19:44:  <!-- Network -->
16:19:44:  <proxy v=':8080'/>
16:19:44:
16:19:44:  <!-- Slot Control -->
16:19:44:  <power v='full'/>
16:19:44:
16:19:44:  <!-- User Information -->
16:19:44:  <passkey v='********************************'/>
16:19:44:  <team v='224497'/>
16:19:44:  <user v='xxx'/>
16:19:44:
16:19:44:  <!-- Folding Slots -->
16:19:44:  <slot id='0' type='GPU'>
16:19:44:    <paused v='true'/>
16:19:44:  </slot>
16:19:44:  <slot id='1' type='GPU'>
16:19:44:    <gpu-index v='1'/>
16:19:44:    <paused v='true'/>
16:19:44:  </slot>
16:19:44:</config>
16:19:44:Trying to access database...
16:19:44:Successfully acquired database lock
16:19:44:Enabled folding slot 00: PAUSED gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 (by user)
16:19:44:Enabled folding slot 01: PAUSED gpu:1:GP102 [GeForce GTX 1080 Ti] 11380 (by user)
...
16:57:19:WU01:FS00:0x21:Completed 4500000 out of 6250000 steps (72%)
16:57:56:WU00:FS01:0x21:Completed 6125000 out of 6250000 steps (98%)
16:58:20:WU01:FS00:0x21:Completed 4562500 out of 6250000 steps (73%)
16:58:49:WU00:FS01:0x21:Completed 6187500 out of 6250000 steps (99%)
16:58:50:WU02:FS01:Connecting to 171.67.108.45:80
16:58:51:WU02:FS01:Assigned to work server 171.67.108.157
16:58:51:WU02:FS01:Requesting new work unit for slot 01: RUNNING gpu:1:GP102 [GeForce GTX 1080 Ti] 11380 from 171.67.108.157
16:58:51:WU02:FS01:Connecting to 171.67.108.157:8080
16:58:52:WU02:FS01:Downloading 5.18MiB
16:58:55:WU02:FS01:Download complete
16:58:55:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:9415 run:950 clone:2 gen:95 core:0x21 unit:0x0000006eab436c9d585e06d126a01eaa
16:59:20:WU01:FS00:0x21:Completed 4625000 out of 6250000 steps (74%)
16:59:42:WU00:FS01:0x21:Completed 6250000 out of 6250000 steps (100%)
16:59:42:WU00:FS01:0x21:Saving result file logfile_01.txt
16:59:42:WU00:FS01:0x21:Saving result file checkpointState.xml
16:59:42:WU00:FS01:0x21:Saving result file checkpt.crc
16:59:42:WU00:FS01:0x21:Saving result file log.txt
16:59:42:WU00:FS01:0x21:Saving result file positions.xtc
16:59:42:WU00:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
16:59:43:WU00:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
16:59:43:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9414 run:2541 clone:0 gen:579 core:0x21 unit:0x000002a1ab436c9d585e06a4b98f2e50
16:59:43:WU00:FS01:Uploading 7.80MiB to 171.67.108.157
16:59:43:WU00:FS01:Connecting to 171.67.108.157:8080
16:59:43:WU02:FS01:Starting
16:59:43:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/fahwebx.stanford.edu/cores/Win32/AMD64/NVIDIA/Fermi/beta/Core_21.fah/FahCore_21.exe -dir 02 -suffix 01 -version 704 -lifeline 3828 -checkpoint 5 -gpu-vendor nvidia -opencl-platform 2 -opencl-device 1 -cuda-device 1 -gpu 1
16:59:43:WU02:FS01:Started FahCore on PID 7264
16:59:43:WU02:FS01:Core PID:9176
16:59:43:WU02:FS01:FahCore 0x21 started
16:59:43:WU02:FS01:0x21:*********************** Log Started 2017-12-19T16:59:43Z ***********************
16:59:43:WU02:FS01:0x21:Project: 9415 (Run 950, Clone 2, Gen 95)
16:59:43:WU02:FS01:0x21:Unit: 0x0000006eab436c9d585e06d126a01eaa
16:59:43:WU02:FS01:0x21:CPU: 0x00000000000000000000000000000000
16:59:43:WU02:FS01:0x21:Machine: 1
16:59:43:WU02:FS01:0x21:Reading tar file core.xml
16:59:43:WU02:FS01:0x21:Reading tar file integrator.xml
16:59:43:WU02:FS01:0x21:Reading tar file state.xml
16:59:43:WU02:FS01:0x21:Reading tar file system.xml
16:59:43:WU02:FS01:0x21:Digital signatures verified
16:59:43:WU02:FS01:0x21:Folding@home GPU Core21 Folding@home Core
16:59:43:WU02:FS01:0x21:Version 0.0.20
16:59:49:WU00:FS01:Upload 4.81%
16:59:55:WU00:FS01:Upload 9.61%
17:00:01:WU00:FS01:Upload 14.42%
17:00:07:WU00:FS01:Upload 20.03%
17:00:13:WU00:FS01:Upload 24.84%
17:00:19:WU00:FS01:Upload 29.64%
17:00:21:WU01:FS00:0x21:Completed 4687500 out of 6250000 steps (75%)
17:00:25:WU00:FS01:Upload 35.25%
17:00:31:WU00:FS01:Upload 40.06%
17:00:37:WU00:FS01:Upload 44.87%
17:00:43:WU00:FS01:Upload 50.48%
17:00:49:WU00:FS01:Upload 55.28%
17:00:55:WU00:FS01:Upload 60.09%
17:01:01:WU00:FS01:Upload 65.70%
17:01:07:WU00:FS01:Upload 70.51%
17:01:13:WU00:FS01:Upload 75.31%
17:01:19:WU00:FS01:Upload 80.92%
17:01:21:WU01:FS00:0x21:Completed 4750000 out of 6250000 steps (76%)
17:01:25:WU00:FS01:Upload 85.73%
17:01:31:WU00:FS01:Upload 90.54%
17:01:37:WU00:FS01:Upload 96.14%
17:01:42:WU00:FS01:Upload complete
17:01:42:WU00:FS01:Server responded WORK_ACK (400)
17:01:42:WU00:FS01:Final credit estimate, 45570.00 points
17:01:42:WU00:FS01:Cleaning up
17:02:14:WARNING:WU02:FS01:FahCore returned: FAILED_3 (255 = 0xff)
17:02:15:WU02:FS01:Starting
17:02:15:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/fahwebx.stanford.edu/cores/Win32/AMD64/NVIDIA/Fermi/beta/Core_21.fah/FahCore_21.exe -dir 02 -suffix 01 -version 704 -lifeline 3828 -checkpoint 5 -gpu-vendor nvidia -opencl-platform 2 -opencl-device 1 -cuda-device 1 -gpu 1
17:02:15:WU02:FS01:Started FahCore on PID 8104
17:02:15:WU02:FS01:Core PID:3128
17:02:15:WU02:FS01:FahCore 0x21 started
17:02:15:WU02:FS01:0x21:*********************** Log Started 2017-12-19T17:02:15Z ***********************
17:02:15:WU02:FS01:0x21:Project: 9415 (Run 950, Clone 2, Gen 95)
17:02:15:WU02:FS01:0x21:Unit: 0x0000006eab436c9d585e06d126a01eaa
17:02:15:WU02:FS01:0x21:CPU: 0x00000000000000000000000000000000
17:02:15:WU02:FS01:0x21:Machine: 1
17:02:15:WU02:FS01:0x21:Reading tar file core.xml
17:02:15:WU02:FS01:0x21:Reading tar file integrator.xml
17:02:15:WU02:FS01:0x21:Reading tar file state.xml
17:02:15:WU02:FS01:0x21:Reading tar file system.xml
17:02:15:WU02:FS01:0x21:Digital signatures verified
17:02:15:WU02:FS01:0x21:Folding@home GPU Core21 Folding@home Core
17:02:15:WU02:FS01:0x21:Version 0.0.20
17:02:16:WU02:FS01:0x21:Completed 0 out of 6250000 steps (0%)
17:02:16:WU02:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
17:02:21:WU01:FS00:0x21:Completed 4812500 out of 6250000 steps (77%)
17:03:10:WU02:FS01:0x21:Completed 62500 out of 6250000 steps (1%)
17:03:18:WU01:FS00:0x21:Completed 4875000 out of 6250000 steps (78%)

[bruce]

@foldy:
It really has nothing to do with the upload. This message is identified as coming from :WU02:FS01: so looking only at those messages allows me to exclude unrelated messages. Nevertheless, FAHCoreWrapper.exe and FahCore_21.exe attempt to start, fail, reattempt to start, and succeed. I've never seen that before ... and the error message isn't particularly helpful.

Was there anything in the Windows Event log at that same time?

Did you try the test recommended by jcoffland a few posts above mine?

Code: Select all

16:58:50:WU02:FS01:Connecting to 171.67.108.45:80
16:58:51:WU02:FS01:Assigned to work server 171.67.108.157
16:58:51:WU02:FS01:Requesting new work unit for slot 01: RUNNING gpu:1:GP102 [GeForce GTX 1080 Ti] 11380 from 171.67.108.157
16:58:51:WU02:FS01:Connecting to 171.67.108.157:8080
16:58:52:WU02:FS01:Downloading 5.18MiB
16:58:55:WU02:FS01:Download complete
16:58:55:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:9415 run:950 clone:2 gen:95 core:0x21 unit:0x0000006eab436c9d585e06d126a01eaa
16:59:43:WU02:FS01:Starting
16:59:43:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/fahwebx.stanford.edu/cores/Win32/AMD64/NVIDIA/Fermi/beta/Core_21.fah/FahCore_21.exe -dir 02 -suffix 01 -version 704 -lifeline 3828 -checkpoint 5 -gpu-vendor nvidia -opencl-platform 2 -opencl-device 1 -cuda-device 1 -gpu 1
16:59:43:WU02:FS01:Started FahCore on PID 7264
16:59:43:WU02:FS01:Core PID:9176
16:59:43:WU02:FS01:FahCore 0x21 started
16:59:43:WU02:FS01:0x21:*********************** Log Started 2017-12-19T16:59:43Z ***********************
16:59:43:WU02:FS01:0x21:Project: 9415 (Run 950, Clone 2, Gen 95)
16:59:43:WU02:FS01:0x21:Unit: 0x0000006eab436c9d585e06d126a01eaa
16:59:43:WU02:FS01:0x21:CPU: 0x00000000000000000000000000000000
16:59:43:WU02:FS01:0x21:Machine: 1
16:59:43:WU02:FS01:0x21:Reading tar file core.xml
16:59:43:WU02:FS01:0x21:Reading tar file integrator.xml
16:59:43:WU02:FS01:0x21:Reading tar file state.xml
16:59:43:WU02:FS01:0x21:Reading tar file system.xml
16:59:43:WU02:FS01:0x21:Digital signatures verified
16:59:43:WU02:FS01:0x21:Folding@home GPU Core21 Folding@home Core
16:59:43:WU02:FS01:0x21:Version 0.0.20
17:02:14:WARNING:WU02:FS01:FahCore returned: FAILED_3 (255 = 0xff)
17:02:15:WU02:FS01:Starting
17:02:15:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/fahwebx.stanford.edu/cores/Win32/AMD64/NVIDIA/Fermi/beta/Core_21.fah/FahCore_21.exe -dir 02 -suffix 01 -version 704 -lifeline 3828 -checkpoint 5 -gpu-vendor nvidia -opencl-platform 2 -opencl-device 1 -cuda-device 1 -gpu 1
17:02:15:WU02:FS01:Started FahCore on PID 8104
17:02:15:WU02:FS01:Core PID:3128
17:02:15:WU02:FS01:FahCore 0x21 started
17:02:15:WU02:FS01:0x21:*********************** Log Started 2017-12-19T17:02:15Z ***********************
17:02:15:WU02:FS01:0x21:Project: 9415 (Run 950, Clone 2, Gen 95)
17:02:15:WU02:FS01:0x21:Unit: 0x0000006eab436c9d585e06d126a01eaa
17:02:15:WU02:FS01:0x21:CPU: 0x00000000000000000000000000000000
17:02:15:WU02:FS01:0x21:Machine: 1
17:02:15:WU02:FS01:0x21:Reading tar file core.xml
17:02:15:WU02:FS01:0x21:Reading tar file integrator.xml
17:02:15:WU02:FS01:0x21:Reading tar file state.xml
17:02:15:WU02:FS01:0x21:Reading tar file system.xml
17:02:15:WU02:FS01:0x21:Digital signatures verified
17:02:15:WU02:FS01:0x21:Folding@home GPU Core21 Folding@home Core
17:02:15:WU02:FS01:0x21:Version 0.0.20
17:02:16:WU02:FS01:0x21:Completed 0 out of 6250000 steps (0%)
17:02:16:WU02:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
17:03:10:WU02:FS01:0x21:Completed 62500 out of 6250000 steps (1%)

@jcoffland
is FAHCoreWrapper re-entrant?

[foldy]

I tried jcoffland's test but that did not hurt FAH.

I got this error message in Windows Event Log: FahCore_21.exe: 0xC0000374

Looks similar to:
viewtopic.php?f=19&t=28709&p=284402&hilit=0xC0000374#p284402
viewtopic.php?f=80&t=30241&p=298404&hilit=0xC0000374#p298404
viewtopic.php?f=96&t=30522&p=299278&hilit=0xC0000374#p299278