Project 9712 run:7 clone:22 gen:74

[davidcoton]

Error and recovery at 63%. Note also a funny in the log at 72%.

Code: Select all

*********************** Log Started 2016-01-16T23:46:25Z ***********************
23:46:25:************************* Folding@home Client *************************
23:46:25:      Website: http://folding.stanford.edu/
23:46:25:    Copyright: (c) 2009-2014 Stanford University
23:46:25:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
23:46:25:         Args: 
23:46:25:       Config: C:/Users/David/AppData/Roaming/FAHClient/config.xml
23:46:25:******************************** Build ********************************
23:46:25:      Version: 7.4.4
23:46:25:         Date: Mar 4 2014
23:46:25:         Time: 20:26:54
23:46:25:      SVN Rev: 4130
23:46:25:       Branch: fah/trunk/client
23:46:25:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
23:46:25:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
23:46:25:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
23:46:25:     Platform: win32 XP
23:46:25:         Bits: 32
23:46:25:         Mode: Release
23:46:25:******************************* System ********************************
23:46:25:          CPU: AMD Athlon(tm) II X4 640 Processor
23:46:25:       CPU ID: AuthenticAMD Family 16 Model 5 Stepping 3
23:46:25:         CPUs: 4
23:46:25:       Memory: 3.12GiB
23:46:25:  Free Memory: 1.56GiB
23:46:25:      Threads: WINDOWS_THREADS
23:46:25:   OS Version: 6.0
23:46:25:  Has Battery: false
23:46:25:   On Battery: false
23:46:25:   UTC Offset: 0
23:46:25:          PID: 4052
23:46:25:          CWD: C:/Users/David/AppData/Roaming/FAHClient
23:46:25:           OS: Windows Vista (TM) Home Premium Service Pack 2
23:46:25:      OS Arch: X86
23:46:25:         GPUs: 1
23:46:25:        GPU 0: NVIDIA:5 GM204 [GeForce GTX 980]
23:46:25:         CUDA: 5.2
23:46:25:  CUDA Driver: 8000
23:46:25:Win32 Service: false
23:46:25:***********************************************************************
23:46:25:<config>
23:46:25:  <!-- Folding Core -->
23:46:25:  <checkpoint v='5'/>
23:46:25:
23:46:25:  <!-- HTTP Server -->
23:46:25:  <allow v='127.0.0.1 192.168.1.0/24'/>
23:46:25:  <deny v='0.0.0.0/0'/>
23:46:25:  <http-addresses v='127.0.0.1:7396 david-ubuntu:7396'/>
23:46:25:
23:46:25:  <!-- Network -->
23:46:25:  <proxy v=':8080'/>
23:46:25:
23:46:25:  <!-- Remote Command Server -->
23:46:25:  <password v='*******'/>
23:46:25:
23:46:25:  <!-- Slot Control -->
23:46:25:  <power v='full'/>
23:46:25:
23:46:25:  <!-- User Information -->
23:46:25:  <passkey v='********************************'/>
23:46:25:  <user v='davidcoton'/>
23:46:25:
23:46:25:  <!-- Web Server -->
23:46:25:  <web-allow v='127.0.0.1 168.192.1.0/24'/>
23:46:25:
23:46:25:  <!-- Folding Slots -->
23:46:25:  <slot id='0' type='CPU'>
23:46:25:    <client-type v='advanced'/>
23:46:25:    <cpus v='3'/>
23:46:25:    <paused v='true'/>
23:46:25:  </slot>
23:46:25:  <slot id='1' type='GPU'>
23:46:25:    <client-type v='advanced'/>
23:46:25:    <max-packet-size v='big'/>
23:46:25:    <paused v='true'/>
23:46:25:  </slot>
23:46:25:</config>

Code: Select all

******************************* Date: 2016-01-23 *******************************
00:59:56:WU01:FS01:Connecting to 171.67.108.45:80
00:59:58:WU01:FS01:Assigned to work server 171.64.65.98
00:59:58:WU01:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GM204 [GeForce GTX 980] from 171.64.65.98
00:59:58:WU01:FS01:Connecting to 171.64.65.98:8080
00:59:59:WU01:FS01:Downloading 7.48MiB
01:00:05:WU01:FS01:Download 5.01%
...
01:03:55:WU01:FS01:Download 99.39%
01:03:56:WU01:FS01:Download complete
01:03:56:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:9712 run:7 clone:22 gen:74 core:0x21 unit:0x000000ffab40416255b9a75294dcb345
01:03:56:WU01:FS01:Starting
01:03:56:WU01:FS01:Running FahCore: "C:\Program Files\FAHClient/FAHCoreWrapper.exe" C:/Users/David/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/x86/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 01 -suffix 01 -version 704 -lifeline 4052 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
01:03:56:WU01:FS01:Started FahCore on PID 13628
01:03:56:WU01:FS01:Core PID:12856
01:03:56:WU01:FS01:FahCore 0x21 started
01:03:57:WU01:FS01:0x21:*********************** Log Started 2016-01-23T01:03:57Z ***********************
01:03:57:WU01:FS01:0x21:Project: 9712 (Run 7, Clone 22, Gen 74)
01:03:57:WU01:FS01:0x21:Unit: 0x000000ffab40416255b9a75294dcb345
01:03:57:WU01:FS01:0x21:CPU: 0x00000000000000000000000000000000
01:03:57:WU01:FS01:0x21:Machine: 1
01:03:57:WU01:FS01:0x21:Reading tar file core.xml
01:03:57:WU01:FS01:0x21:Reading tar file integrator.xml
01:03:57:WU01:FS01:0x21:Reading tar file system.xml
01:04:00:WU01:FS01:0x21:Reading tar file state.xml
01:04:02:WU01:FS01:0x21:Digital signatures verified
01:04:02:WU01:FS01:0x21:Folding@home GPU Core21 Folding@home Core
01:04:02:WU01:FS01:0x21:Version 0.0.17
01:05:49:WU01:FS01:0x21:Completed 0 out of 1280000 steps (0%)
01:05:50:WU01:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
01:07:49:WU01:FS01:0x21:Completed 12800 out of 1280000 steps (1%)
01:09:22:WU01:FS01:0x21:Completed 25600 out of 1280000 steps (2%)
...
02:49:23:WU01:FS01:0x21:Completed 793600 out of 1280000 steps (62%)
02:51:36:WU01:FS01:0x21:Completed 806400 out of 1280000 steps (63%)
02:52:36:WARNING:WU01:FS01:FahCore returned an unknown error code which probably indicates that it crashed
02:52:36:WARNING:WU01:FS01:FahCore returned: UNKNOWN_ENUM (127 = 0x7f)
02:52:36:WU01:FS01:Starting
02:52:36:WU01:FS01:Running FahCore: "C:\Program Files\FAHClient/FAHCoreWrapper.exe" C:/Users/David/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/x86/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 01 -suffix 01 -version 704 -lifeline 4052 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
02:52:36:WU01:FS01:Started FahCore on PID 10656
02:52:37:WU01:FS01:Core PID:9448
02:52:37:WU01:FS01:FahCore 0x21 started
02:52:39:WU01:FS01:0x21:*********************** Log Started 2016-01-23T02:52:38Z ***********************
02:52:39:WU01:FS01:0x21:Project: 9712 (Run 7, Clone 22, Gen 74)
02:52:39:WU01:FS01:0x21:Unit: 0x000000ffab40416255b9a75294dcb345
02:52:39:WU01:FS01:0x21:CPU: 0x00000000000000000000000000000000
02:52:39:WU01:FS01:0x21:Machine: 1
02:52:39:WU01:FS01:0x21:Digital signatures verified
02:52:39:WU01:FS01:0x21:Folding@home GPU Core21 Folding@home Core
02:52:39:WU01:FS01:0x21:Version 0.0.17
02:52:39:WU01:FS01:0x21:  Found a checkpoint file
02:54:22:WU01:FS01:0x21:Completed 800000 out of 1280000 steps (62%)
02:54:33:WU01:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
02:55:48:WU01:FS01:0x21:Completed 806400 out of 1280000 steps (63%)
02:57:23:WU01:FS01:0x21:Completed 819200 out of 1280000 steps (64%)
...
03:09:12:WU01:FS01:0x21:Completed 908800 out of 1280000 steps (71%)
03:10:47:WU01:FS01:0x21:C
03:12:28:WU01:FS01:0x21:Completed 934400 out of 1280000 steps (73%)
...
03:56:19:WU01:FS01:0x21:Completed 1267200 out of 1280000 steps (99%)
03:57:55:WU01:FS01:0x21:Completed 1280000 out of 1280000 steps (100%)
03:58:26:WU01:FS01:0x21:Saving result file logfile_01.txt
03:58:27:WU01:FS01:0x21:Saving result file checkpointState.xml
03:58:32:WU01:FS01:0x21:Saving result file checkpt.crc
03:58:32:WU01:FS01:0x21:Saving result file log.txt
03:58:32:WU01:FS01:0x21:Saving result file positions.xtc
03:58:34:WU01:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
03:58:35:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
03:58:35:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9712 run:7 clone:22 gen:74 core:0x21 unit:0x000000ffab40416255b9a75294dcb345
03:58:35:WU01:FS01:Uploading 9.52MiB to 171.64.65.98
03:58:35:WU01:FS01:Connecting to 171.64.65.98:8080
03:58:41:WU01:FS01:Upload 8.53%
...
04:00:05:WU01:FS01:Upload 95.85%
04:00:28:WU01:FS01:Upload complete
04:00:28:WU01:FS01:Server responded WORK_ACK (400)
04:00:28:WU01:FS01:Final credit estimate, 34460.00 points
04:00:28:WU01:FS01:Cleaning up

[Ricky]

davidcoton,

I have seen both of these, but I am not sure that it was in the same work unit. I also notice that the partial log line would not get fixed when I would refresh.

[bruce]

The "funny in the log at 72%" has been seen before and has been reported.

It's priority is probably too low for us to expect a fix any time soon, given FAHs development backlog.

[davidcoton]

The "funny in the log at 72%" has been seen before and has been reported.

It's priority is probably too low for us to expect a fix any time soon, given FAHs development backlog.

Only probably? I wouldn't have mentioned it except that it was in the same bit of log as the other error.
Which itself is probably not significant on a post-beta WU.

[bruce]

My hunch: It has nothing to do with it being a post-beta WU. It's a client bug based on the number of bytes in the log.

[davidcoton]

I agree nothing to do with post-beta. Just that since it is out of the test phase no-one is going to pick up on a non-fatal error -- unless it happens often enough to start trashing WUs. In beta it might just be worth reporting, but not afterwards.

[bruce]

My hunch (again) is that they'll never see it. The problem only occurs in the log that you and I look at (with the time-stamps, the Slot number, and the WU number. They probably only look at a log when they need to and then they'll look at a more detailed one.