Moderators: Site Moderators , FAHC Science Team
PawelS
Posts: 10 Joined: Fri Nov 08, 2019 2:23 pm
Post
by PawelS Fri Nov 08, 2019 3:21 pm
This info I've found in a log file after deleting the folder of the stuck CPU slot - I had thought it was a 1-time case perhaps after Windows crash by automatic backup service what happened from time to time. After the crash, I saw previous CPU WU was rejected by a server and a new WU was much bigger than anything before. I started folding nearly 3 weeks before. CPU was running without progress and WU couldn't be effectively finished by clicking. It is the same after deleting the WU folder. The newest WU is from the same project but in the folder, there is no other file than wudata_01.dat.
The old log file from deleted WU folder:
Code: Select all
Author: Joseph Coffland <joseph@cauldrondevelopment.com>
Args: -dir 02 -suffix 01 -version 705 -lifeline 4284 -checkpoint 15 -np 2
Config: <none>
************************************ Build *************************************
Version: 0.0.17
Date: Apr 27 2018
Time: 16:19:36
Repository: Git
Revision: 21359963583d09ec2063ef946399441c4df4ccd7
Branch: master
Compiler: Visual C++ 2008
Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
Platform: win32 10
Bits: 64
Mode: Release
SIMD: avx_256
************************************ System ************************************
CPU: Unknown
CPU ID:
CPUs: 4
Memory: 14.92GiB
Free Memory: 10.52GiB
Threads: WINDOWS_THREADS
OS Version: 6.2
Has Battery: false
On Battery: false
UTC Offset: 0
PID: 3476
CWD: C:\Users\pawci\AppData\Roaming\FAHClient\work
OS: Windows 10 Home
OS Arch: AMD64
********************************************************************************
Project: 14183 (Run 0, Clone 368, Gen 14)
Unit: 0x0000000f0002894b5dae083bb04fa997
ERROR:110: Need version 0.0.18
Folding@home Core Shutdown: CORE_OUTDATED
The current log from Client Advanced Control
Code: Select all
13:15:45: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
13:15:45: Args:
13:15:45: Config: C:\Users\pawci\AppData\Roaming\FAHClient\config.xml
13:15:45:******************************** Build ********************************
13:15:45: Version: 7.5.1
13:15:45: Date: May 11 2018
13:15:45: Time: 13:06:32
13:15:45: Repository: Git
13:15:45: Revision: 4705bf53c635f88b8fe85af7675557e15d491ff0
13:15:45: Branch: master
13:15:45: Compiler: Visual C++ 2008
13:15:45: Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
13:15:45: Platform: win32 10
13:15:45: Bits: 32
13:15:45: Mode: Release
13:15:45:******************************* System ********************************
13:15:45: CPU: AMD A8-7600 Radeon R7, 10 Compute Cores 4C+6G
13:15:45: CPU ID: AuthenticAMD Family 21 Model 48 Stepping 1
13:15:45: CPUs: 4
13:15:45: Memory: 14.92GiB
13:15:45: Free Memory: 11.47GiB
13:15:45: Threads: WINDOWS_THREADS
13:15:45: OS Version: 6.2
13:15:45: Has Battery: false
13:15:45: On Battery: false
13:15:45: UTC Offset: 0
13:15:45: PID: 7252
13:15:45: CWD: C:\Users\pawci\AppData\Roaming\FAHClient
13:15:45: OS: Windows 10 Home
13:15:45: OS Arch: AMD64
13:15:45: GPUs: 1
13:15:45: GPU 0: Bus:0 Slot:1 Func:0 AMD:5 Kaveri [Radeon R7]
13:15:45: CUDA: Not detected: Failed to open dynamic library 'nvcuda.dll': Nie
13:15:45: można odnaleźć określonego modułu.
13:15:45:
13:15:45:OpenCL Device 0: Platform:0 Device:0 Bus:0 Slot:1 Compute:1.2 Driver:2841.5
13:15:45: Win32 Service: false
13:15:45:***********************************************************************
13:15:45:<config>
13:15:45: <!-- Slot Control -->
13:15:45: <power v='full'/>
13:15:45:
13:15:45: <!-- User Information -->
13:15:45: <passkey v='********************************'/>
13:15:45: <user v='PawelS'/>
13:15:45:
13:15:45: <!-- Folding Slots -->
13:15:45: <slot id='0' type='CPU'/>
13:15:45: <slot id='1' type='GPU'/>
13:15:45:</config>
13:15:45:Trying to access database...
13:15:45:Successfully acquired database lock
13:15:45:Enabled folding slot 00: READY cpu:3
13:15:45:Enabled folding slot 01: READY gpu:0:Kaveri [Radeon R7]
13:15:45:WU01:FS00:Starting
13:15:45:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\pawci\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/v7/win/64bit/avx/Core_a7.fah/FahCore_a7.exe -dir 01 -suffix 01 -version 705 -lifeline 7252 -checkpoint 15 -np 3
13:15:45:WU01:FS00:Started FahCore on PID 6660
13:15:47:WU01:FS00:Core PID:11864
13:15:47:WU01:FS00:FahCore 0xa7 started
13:15:47:WU00:FS01:Starting
13:15:47:WU00:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\pawci\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/v7/win/64bit/amd/Core_21.fah/FahCore_21.exe -dir 00 -suffix 01 -version 705 -lifeline 7252 -checkpoint 15 -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
13:15:47:WU00:FS01:Started FahCore on PID 8004
13:15:47:WU00:FS01:Core PID:840
13:15:47:WU00:FS01:FahCore 0x21 started
13:15:50:WU00:FS01:0x21:*********************** Log Started 2019-11-08T13:15:49Z ***********************
13:15:50:WU00:FS01:0x21:Project: 14196 (Run 13, Clone 471, Gen 22)
13:15:50:WU00:FS01:0x21:Unit: 0x0000002c0002894b5d92468ab04534e4
13:15:50:WU00:FS01:0x21:CPU: 0x00000000000000000000000000000000
13:15:50:WU00:FS01:0x21:Machine: 1
13:15:50:WU00:FS01:0x21:Digital signatures verified
13:15:50:WU00:FS01:0x21:Folding@home GPU Core21 Folding@home Core
13:15:50:WU00:FS01:0x21:Version 0.0.20
13:15:50:WU00:FS01:0x21: Found a checkpoint file
13:16:25:WU00:FS01:0x21:Completed 3750000 out of 5000000 steps (75%)
13:16:25:WU00:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
14:01:54:WU00:FS01:0x21:Completed 3800000 out of 5000000 steps (76%)
14:13:11:FS00:Paused
14:13:11:FS00:Shutting core down
14:13:12:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
14:13:18:FS00:Unpaused
14:13:18:WU01:FS00:Starting
14:13:18:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\pawci\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/v7/win/64bit/avx/Core_a7.fah/FahCore_a7.exe -dir 01 -suffix 01 -version 705 -lifeline 7252 -checkpoint 15 -np 3
14:13:18:WU01:FS00:Started FahCore on PID 1120
14:13:18:WU01:FS00:Core PID:3248
14:13:18:WU01:FS00:FahCore 0xa7 started
14:14:01:FS00:Finishing
14:48:46:WU00:FS01:0x21:Completed 3850000 out of 5000000 steps (77%)
bruce
Posts: 20910 Joined: Thu Nov 29, 2007 10:13 pmLocation: So. Cal.
Post
by bruce Fri Nov 08, 2019 3:40 pm
Your post is missing some critical information or at least I don't understand what I see.
ComputerUser
Posts: 27 Joined: Thu Oct 31, 2019 7:58 am
Post
by ComputerUser Fri Nov 08, 2019 3:47 pm
Hi PawelS,
just for reference, this is what it should look like:
Code: Select all
15:35:43:WU00:FS00:Connecting to 65.254.110.245:8080
15:35:44:WU00:FS00:Assigned to work server 128.252.203.10
15:35:44:WU00:FS00:Requesting new work unit for slot 00: READY cpu:2 from 128.252.203.10
15:35:44:WU00:FS00:Connecting to 128.252.203.10:8080
15:35:45:WU00:FS00:Downloading 1.17MiB
15:35:46:WU00:FS00:Download complete
15:35:46:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:14246 run:0 clone:30 gen:105 core:0xa7 unit:0x0000009580fccb0a5d6fe222adf8193b
15:35:46:WU00:FS00:Starting
15:35:46:WU00:FS00:Running FahCore: "C:\Program Files\FAHClient/FAHCoreWrapper.exe" C:/Users/[redacted]/AppData/Roaming/FAHClient/cores/cores.foldingathome.org/v7/win/32bit/Core_a7.fah/FahCore_a7.exe -dir 00 -suffix 01 -version 704 -lifeline 4564 -checkpoint 15 -np 2
15:35:46:WU00:FS00:Started FahCore on PID 7876
15:35:47:Started thread 29 on PID 4564
15:35:47:WU00:FS00:Core PID:5428
15:35:47:WU00:FS00:FahCore 0xa7 started
15:35:49:WU00:FS00:0xa7:*********************** Log Started 2019-11-08T15:35:48Z ***********************
15:35:49:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
15:35:49:WU00:FS00:0xa7: Type: 0xa7
15:35:49:WU00:FS00:0xa7: Core: Gromacs
15:35:49:WU00:FS00:0xa7: Website: https://foldingathome.org/
15:35:49:WU00:FS00:0xa7: Copyright: (c) 2009-2018 foldingathome.org
15:35:49:WU00:FS00:0xa7: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
15:35:49:WU00:FS00:0xa7: Args: -dir 00 -suffix 01 -version 704 -lifeline 7876 -checkpoint 15 -np 2
15:35:49:WU00:FS00:0xa7: Config: <none>
15:35:49:WU00:FS00:0xa7:************************************ Build *************************************
15:35:49:WU00:FS00:0xa7: Version: 0.0.17
15:35:49:WU00:FS00:0xa7: Date: Apr 25 2018
15:35:49:WU00:FS00:0xa7: Time: 11:02:26
15:35:49:WU00:FS00:0xa7: Repository: Git
15:35:49:WU00:FS00:0xa7: Revision: fd11abfb405c921e66db1226933e9dd2d18d2acc
15:35:49:WU00:FS00:0xa7: Branch: master
15:35:49:WU00:FS00:0xa7: Compiler: Visual C++ 2008
15:35:49:WU00:FS00:0xa7: Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
15:35:49:WU00:FS00:0xa7: Platform: win32 10
15:35:49:WU00:FS00:0xa7: Bits: 32
15:35:49:WU00:FS00:0xa7: Mode: Release
15:35:49:WU00:FS00:0xa7: SIMD: sse2
15:35:49:WU00:FS00:0xa7:************************************ System ************************************
15:35:49:WU00:FS00:0xa7: CPU: Unknown
15:35:49:WU00:FS00:0xa7: CPU ID:
15:35:49:WU00:FS00:0xa7: CPUs: 2
15:35:49:WU00:FS00:0xa7: Memory: 3.00GiB
15:35:49:WU00:FS00:0xa7:Free Memory: 886.70MiB
15:35:49:WU00:FS00:0xa7: Threads: WINDOWS_THREADS
15:35:49:WU00:FS00:0xa7: OS Version: 6.1
15:35:49:WU00:FS00:0xa7:Has Battery: false
15:35:49:WU00:FS00:0xa7: On Battery: false
15:35:49:WU00:FS00:0xa7: UTC Offset: 1
15:35:49:WU00:FS00:0xa7: PID: 5428
15:35:49:WU00:FS00:0xa7: CWD: C:\Users\[redacted]\AppData\Roaming\FAHClient\work
15:35:49:WU00:FS00:0xa7: OS: Windows 7 Professional Service Pack 1
15:35:49:WU00:FS00:0xa7: OS Arch: X86
15:35:49:WU00:FS00:0xa7:********************************************************************************
15:35:49:WU00:FS00:0xa7:Project: 14246 (Run 0, Clone 30, Gen 105)
15:35:49:WU00:FS00:0xa7:Unit: 0x0000009580fccb0a5d6fe222adf8193b
15:35:49:WU00:FS00:0xa7:ERROR:110: Need version 0.0.18
15:35:49:WU00:FS00:0xa7:Folding@home Core Shutdown: CORE_OUTDATED
15:35:49:WARNING:WU00:FS00:FahCore returned: CORE_OUTDATED (110 = 0x6e)
15:35:50:WU00:FS00:Downloading core from http://cores.foldingathome.org/v7/win/32bit/Core_a7.fah
15:35:50:WU00:FS00:Connecting to cores.foldingathome.org:80
15:35:51:WU00:FS00:FahCore a7: Downloading 5.20MiB
15:35:54:WU00:FS00:FahCore a7: Download complete
15:35:54:WU00:FS00:Valid core signature
15:35:55:WU00:FS00:Unpacked 15.50MiB to cores/cores.foldingathome.org/v7/win/32bit/Core_a7.fah/FahCore_a7.exe
15:35:55:WU00:FS00:Unpacked 2.07MiB to cores/cores.foldingathome.org/v7/win/32bit/Core_a7.fah/libfftw3f-3.dll
15:35:55:WU00:FS00:Unpacked 105.50KiB to cores/cores.foldingathome.org/v7/win/32bit/Core_a7.fah/libgcc_s_dw2-1.dll
15:35:55:WU00:FS00:Starting
15:35:55:WU00:FS00:Running FahCore: "C:\Program Files\FAHClient/FAHCoreWrapper.exe" C:/Users/[redacted]/AppData/Roaming/FAHClient/cores/cores.foldingathome.org/v7/win/32bit/Core_a7.fah/FahCore_a7.exe -dir 00 -suffix 01 -version 704 -lifeline 4564 -checkpoint 15 -np 2
15:35:55:WU00:FS00:Started FahCore on PID 5044
15:35:55:Started thread 30 on PID 4564
15:35:58:WU00:FS00:Core PID:6520
15:35:58:WU00:FS00:FahCore 0xa7 started
15:35:59:WU00:FS00:0xa7:*********************** Log Started 2019-11-08T15:35:58Z ***********************
15:35:59:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
15:35:59:WU00:FS00:0xa7: Type: 0xa7
15:35:59:WU00:FS00:0xa7: Core: Gromacs
15:35:59:WU00:FS00:0xa7: Args: -dir 00 -suffix 01 -version 704 -lifeline 5044 -checkpoint 15 -np 2
15:35:59:WU00:FS00:0xa7:************************************ CBang *************************************
15:35:59:WU00:FS00:0xa7: Date: Oct 26 2019
15:35:59:WU00:FS00:0xa7: Time: 01:27:58
15:35:59:WU00:FS00:0xa7: Revision: c46a1a011a24143739ac7218c5a435f66777f62f
15:35:59:WU00:FS00:0xa7: Branch: master
15:35:59:WU00:FS00:0xa7: Compiler: Visual C++ 2008
15:35:59:WU00:FS00:0xa7: Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
15:35:59:WU00:FS00:0xa7: Platform: win32 10
15:35:59:WU00:FS00:0xa7: Bits: 32
15:35:59:WU00:FS00:0xa7: Mode: Release
15:35:59:WU00:FS00:0xa7:************************************ System ************************************
15:35:59:WU00:FS00:0xa7: CPU: Pentium(R) Dual-Core CPU E5800 @ 3.20GHz
15:35:59:WU00:FS00:0xa7: CPU ID: GenuineIntel Family 6 Model 23 Stepping 10
15:35:59:WU00:FS00:0xa7: CPUs: 2
15:35:59:WU00:FS00:0xa7: Memory: 3.00GiB
15:35:59:WU00:FS00:0xa7:Free Memory: 878.32MiB
15:35:59:WU00:FS00:0xa7: Threads: WINDOWS_THREADS
15:35:59:WU00:FS00:0xa7: OS Version: 6.1
15:35:59:WU00:FS00:0xa7:Has Battery: false
15:35:59:WU00:FS00:0xa7: On Battery: false
15:35:59:WU00:FS00:0xa7: UTC Offset: 1
15:35:59:WU00:FS00:0xa7: PID: 6520
15:35:59:WU00:FS00:0xa7: CWD: C:\Users[redacted]\AppData\Roaming\FAHClient\work
15:35:59:WU00:FS00:0xa7:******************************** Build - libFAH ********************************
15:35:59:WU00:FS00:0xa7: Version: 0.0.18
15:35:59:WU00:FS00:0xa7: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
15:35:59:WU00:FS00:0xa7: Copyright: 2019 foldingathome.org
15:35:59:WU00:FS00:0xa7: Homepage: https://foldingathome.org/
15:35:59:WU00:FS00:0xa7: Date: Oct 26 2019
15:35:59:WU00:FS00:0xa7: Time: 01:34:37
15:35:59:WU00:FS00:0xa7: Revision: c1e3513b1bc0c16013668f2173ee969e5995b38e
15:35:59:WU00:FS00:0xa7: Branch: master
15:35:59:WU00:FS00:0xa7: Compiler: Visual C++ 2008
15:35:59:WU00:FS00:0xa7: Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
15:35:59:WU00:FS00:0xa7: Platform: win32 10
15:35:59:WU00:FS00:0xa7: Bits: 32
15:35:59:WU00:FS00:0xa7: Mode: Release
15:35:59:WU00:FS00:0xa7:************************************ Build *************************************
15:35:59:WU00:FS00:0xa7: SIMD: sse2
15:35:59:WU00:FS00:0xa7:********************************************************************************
15:35:59:WU00:FS00:0xa7:Project: 14246 (Run 0, Clone 30, Gen 105)
15:35:59:WU00:FS00:0xa7:Unit: 0x0000009580fccb0a5d6fe222adf8193b
15:35:59:WU00:FS00:0xa7:Reading tar file core.xml
15:35:59:WU00:FS00:0xa7:Reading tar file frame105.tpr
15:35:59:WU00:FS00:0xa7:Digital signatures verified
15:35:59:WU00:FS00:0xa7:Calling: mdrun -s frame105.tpr -o frame105.trr -x frame105.xtc -cpt 15 -nt 2
15:35:59:WU00:FS00:0xa7:Steps: first=26250000 total=250000
15:36:00:WU00:FS00:0xa7:Completed 1 out of 250000 steps (0%)
15:39:11:WU00:FS00:0xa7:Completed 2500 out of 250000 steps (1%)
15:42:13:WU00:FS00:0xa7:Completed 5000 out of 250000 steps (2%)
Are there any more lines after "Folding@home Core Shutdown: CORE_OUTDATED"?
Computeruser
bruce
Posts: 20910 Joined: Thu Nov 29, 2007 10:13 pmLocation: So. Cal.
Post
by bruce Fri Nov 08, 2019 3:58 pm
The current log that you posted shows that the new core was successfully update, which is what I expected. The science log that you found only contains information from a single invocation of the FAHCore, so the one you posted ends with the OUTDATED message. A new science log would be found elsewhere, showing that the WU did start using the new core and probably was successfully completed. (I can tell that from the current log.)
PawelS
Posts: 10 Joined: Fri Nov 08, 2019 2:23 pm
Post
by PawelS Fri Nov 08, 2019 5:12 pm
ComputerUser wrote: Are there any more lines after "Folding@home Core Shutdown: CORE_OUTDATED"?
No.
Bruce wrote: FAHCore_a7, version 0.0.17 is being updated to version 0.0.18 to fix a known problem (discussed elsewhere). That information should be in the current log but I don't see it. The science log posted above (from the WU's working files) does report that that event did happen but that probably caused your system to simply download the new version of the core and then resume processing. Instead, I see lots of information about FAHCore_21, so maybe you selected the wrong WU and the files simply do not match.
I deleted the folder for stuck CPU WU because GPU folding continues after this.
FAHControl does allow you to view the current log filtered either by Slot or by WU.
I haven't filter the log from FAHClient.
A new science log would be found elsewhere, showing that the WU did start using the new core and probably was successfully completed.
To make sure is a science log a file log in a WU folder? Still \AppData\Roaming\FAHClient\work\01 contains only wudata_01.dat
So maybe Windows crash has damaged something? What should I do?
bruce
Posts: 20910 Joined: Thu Nov 29, 2007 10:13 pmLocation: So. Cal.
Post
by bruce Fri Nov 08, 2019 5:25 pm
The main log (the ones with the timestamps) generally tells you everything about the operation of your client. The science log is attached to the WU when it's uploaded and it is a reasonable source of information intended to help the person running the project if (s)he needs to debug something in their project. The log saying the core needed to be updated wasn't important and probably wasn't even uploaded. That log rarely contains any information that would be helpful to you ... and it had no information about whatever windows crash you had. It's highly unlikely that the damaged FAH results reached a server. That WU either resumed work from the last checkpoint or they were simply dumped and reassigned to someone else. FAH does an excellent job of gathering accurate science.