Projects 762x Testing Core v2.25 on Adv

[JimF]

Well I just received a GTX 660 yesterday, and it has been happily Folding 8072's at 83% power load and 65 C temperatures. But this morning I got a 7624 (R66, C0, G121), and it runs at 96% (even up to 98%) power load (TDP as measured by GPU-Z), and temps of 70 C. And the really curious thing is that the boost clock is now 1110 MHz, which is above the 1059 MHz specified for the card, and I have seen it up to 1123 MHz.

That is no problem at all; I got the card to help heat the room to 68 F, and am glad to see it utilized to its maximum; I am getting what I paid for.
But overclocking (which I don't do anyway) is not for these cards now, so be careful.

EDIT: I forgot to mention that I am now getting 27,058 PPD for the GTX 660 on P7624, which is OK with me. For P7623s on my GTX 560 I am getting 23,872 PPD, and on a GTX 560 Ti am getting 26,412 PPD. But yesterday on the 8072's the GTX 660 was between the 560 and the 560 Ti in points. So it seems that the harder the work, the better the GTX 660 does, relative to the older cards.

[Jesse_V]

I got a number of EUEs with these on my GTX 560TI, and the slot failed and shut down. I recently got the card back from the manufacturer (the fans were stiff so it was previously having serious cooling issues) but I don't think my recent installation is related.

Build/System/Config:

Code: Select all

*********************** Log Started 2013-01-07T01:43:16Z ***********************
01:43:16:************************* Folding@home Client *************************
01:43:16:      Website: http://folding.stanford.edu/
01:43:16:    Copyright: (c) 2009-2012 Stanford University
01:43:16:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
01:43:16:         Args: --lifeline 3200 --command-port=36330
01:43:16:       Config: C:/Users/Admin/AppData/Roaming/FAHClient/config.xml
01:43:16:******************************** Build ********************************
01:43:16:      Version: 7.2.9
01:43:16:         Date: Oct 3 2012
01:43:16:         Time: 18:05:48
01:43:16:      SVN Rev: 3578
01:43:16:       Branch: fah/trunk/client
01:43:16:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
01:43:16:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
01:43:16:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
01:43:16:     Platform: win32 XP
01:43:16:         Bits: 32
01:43:16:         Mode: Release
01:43:16:******************************* System ********************************
01:43:16:          CPU: Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
01:43:16:       CPU ID: GenuineIntel Family 6 Model 42 Stepping 7
01:43:16:         CPUs: 8
01:43:16:       Memory: 7.98GiB
01:43:16:  Free Memory: 6.45GiB
01:43:16:      Threads: WINDOWS_THREADS
01:43:16:   On Battery: false
01:43:16:   UTC offset: -7
01:43:16:          PID: 3544
01:43:16:          CWD: C:/Users/Admin/AppData/Roaming/FAHClient
01:43:16:           OS: Windows 7 Professional N
01:43:16:      OS Arch: AMD64
01:43:16:         GPUs: 1
01:43:16:        GPU 0: NVIDIA:2 GF114 [GeForce GTX 560 Ti]
01:43:16:         CUDA: 2.1
01:43:16:  CUDA Driver: 5000
01:43:16:Win32 Service: false
01:43:16:***********************************************************************
01:43:16:<config>
01:43:16:  <!-- FahCore Control -->
01:43:16:  <checkpoint v='30'/>
01:43:16:  <cpu-affinity v='true'/>
01:43:16:
01:43:16:  <!-- Folding Slot Configuration -->
01:43:16:  <client-type v='beta'/>
01:43:16:  <gpu v='true'/>
01:43:16:
01:43:16:  <!-- Network -->
01:43:16:  <proxy v=':8080'/>
01:43:16:
01:43:16:  <!-- Remote Command Server -->
01:43:16:  <command-allow v='129.123.0.0-129.123.255.255'/>
01:43:16:  <command-deny v=''/>
01:43:16:  <password v='******'/>
01:43:16:
01:43:16:  <!-- User Information -->
01:43:16:  <passkey v='********************************'/>
01:43:16:  <team v='195965'/>
01:43:16:  <user v='Jesse_Victors'/>
01:43:16:
01:43:16:  <!-- Folding Slots -->
01:43:16:  <slot id='1' type='SMP'/>
01:43:16:  <slot id='0' type='GPU'/>
01:43:16:</config>

Errors:

Code: Select all

*********************** Log Started 2013-01-07T01:43:16Z ***********************
01:43:17:WU01:FS00:Starting
01:43:17:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 01 -suffix 01 -version 702 -lifeline 3544 -checkpoint 30 -gpu 0
01:43:17:WU01:FS00:Started FahCore on PID 3788
01:43:18:WU01:FS00:Core PID:3896
01:43:18:WU01:FS00:FahCore 0x15 started
01:43:18:WU01:FS00:0x15:
01:43:18:WU01:FS00:0x15:*------------------------------*
01:43:18:WU01:FS00:0x15:Folding@Home GPU Core
01:43:18:WU01:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
01:43:18:WU01:FS00:0x15:Build host             AmoebaRemote
01:43:18:WU01:FS00:0x15:Board Type             NVIDIA/CUDA
01:43:18:WU01:FS00:0x15:Core                   15
01:43:18:WU01:FS00:0x15:
01:43:18:WU01:FS00:0x15:Window's signal control handler registered.
01:43:18:WU01:FS00:0x15:Preparing to commence simulation
01:43:18:WU01:FS00:0x15:- Ensuring status. Please wait.
01:43:28:WU01:FS00:0x15:- Looking at optimizations...
01:43:28:WU01:FS00:0x15:- Working with standard loops on this execution.
01:43:28:WU01:FS00:0x15:- Previous termination of core was improper.
01:43:28:WU01:FS00:0x15:- Files status OK
01:43:28:WU01:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
01:43:28:WU01:FS00:0x15:- Expanded 124599 -> 501826 (decompressed 402.7 percent)
01:43:28:WU01:FS00:0x15:Called DecompressByteArray: compressed_data_size=124599 data_size=501826, decompressed_data_size=501826 diff=0
01:43:28:WU01:FS00:0x15:- Digital signature verified
01:43:28:WU01:FS00:0x15:
01:43:28:WU01:FS00:0x15:Project: 7623 (Run 93, Clone 7, Gen 11)
01:43:28:WU01:FS00:0x15:
01:43:28:WU01:FS00:0x15:Entering M.D.
01:43:30:WU01:FS00:0x15:Tpr hash 01/wudata_01.tpr:  479633058 1360939802 3843960003 3755814914 2322957242
01:43:30:WU01:FS00:0x15:GPU device id=0
01:43:30:WU01:FS00:0x15:Working on Protein
01:43:30:WU01:FS00:0x15:Client config unavailable.
01:43:30:WU01:FS00:0x15:Starting GUI Server
01:44:31:WU01:FS00:0x15:Setting checkpoint frequency: 400000
01:44:31:WU01:FS00:0x15:Completed         3 out of 40000000 steps (0%).
01:44:32:WARNING:WU01:FS00:Detected clock skew (1 mins 05 secs), adjusting time estimates
01:52:20:WU01:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
02:00:07:WU01:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
02:07:56:WU01:FS00:0x15:Completed   1200000 out of 40000000 steps (3%).
02:15:43:WU01:FS00:0x15:Completed   1600000 out of 40000000 steps (4%).
02:23:30:WU01:FS00:0x15:Completed   2000000 out of 40000000 steps (5%).
02:31:17:WU01:FS00:0x15:Completed   2400000 out of 40000000 steps (6%).
02:39:04:WU01:FS00:0x15:Completed   2800000 out of 40000000 steps (7%).
02:42:15:WU01:FS00:0x15:Run: exception thrown in GuardedRun -- cannot continue further.
02:42:15:WU01:FS00:0x15:Going to send back what have done -- stepsTotalG=40000000
02:42:15:WU01:FS00:0x15:Work fraction=0.0741 steps=40000000.
02:42:19:WU01:FS00:0x15:logfile size=14556 infoLength=14556 edr=0 trr=23
02:42:19:WU01:FS00:0x15:+ Opened results file
02:42:19:WU01:FS00:0x15:- Writing 15092 bytes of core data to disk...
02:42:19:WU01:FS00:0x15:Done: 14580 -> 4762 (compressed to 32.6 percent)
02:42:19:WU01:FS00:0x15:  ... Done.
02:42:19:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
02:42:19:WU01:FS00:0x15:
02:42:19:WU01:FS00:0x15:Folding@home Core Shutdown: EARLY_UNIT_END
02:42:20:WARNING:WU01:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
02:42:20:WU01:FS00:Sending unit results: id:01 state:SEND error:FAULTY project:7623 run:93 clone:7 gen:11 core:0x15 unit:0x0000000b664f2dd14fe4f9008f06fe66
02:42:20:WU01:FS00:Uploading 5.15KiB to 171.64.65.105
02:42:20:WU01:FS00:Connecting to 171.64.65.105:8080
02:42:20:WU02:FS00:Connecting to assign-GPU.stanford.edu:80
02:42:20:WU01:FS00:Upload complete
02:42:20:WU01:FS00:Server responded WORK_ACK (400)
02:42:20:WU01:FS00:Cleaning up
02:42:20:WU02:FS00:News: Welcome to Folding@Home
02:42:20:WU02:FS00:Assigned to work server 171.64.65.105
02:42:20:WU02:FS00:Requesting new work unit for slot 00: READY gpu:0:"GF114 [GeForce GTX 560 Ti]" from 171.64.65.105
02:42:20:WU02:FS00:Connecting to 171.64.65.105:8080
02:42:20:WU02:FS00:Downloading 122.22KiB
02:42:21:WU02:FS00:Download complete
02:42:21:WU02:FS00:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:7623 run:577 clone:0 gen:31 core:0x15 unit:0x0000002e664f2dd14fe4fbaed5c603f0
02:42:21:WU02:FS00:Starting
02:42:21:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 702 -lifeline 3544 -checkpoint 30 -gpu 0
02:42:21:WU02:FS00:Started FahCore on PID 3596
02:42:21:WU02:FS00:Core PID:3404
02:42:21:WU02:FS00:FahCore 0x15 started
02:42:21:WU02:FS00:0x15:
02:42:21:WU02:FS00:0x15:*------------------------------*
02:42:21:WU02:FS00:0x15:Folding@Home GPU Core
02:42:21:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
02:42:21:WU02:FS00:0x15:Build host             AmoebaRemote
02:42:21:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
02:42:21:WU02:FS00:0x15:Core                   15
02:42:21:WU02:FS00:0x15:
02:42:21:WU02:FS00:0x15:Window's signal control handler registered.
02:42:21:WU02:FS00:0x15:Preparing to commence simulation
02:42:21:WU02:FS00:0x15:- Looking at optimizations...
02:42:21:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
02:42:21:WU02:FS00:0x15:- Created dyn
02:42:21:WU02:FS00:0x15:- Files status OK
02:42:21:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
02:42:21:WU02:FS00:0x15:- Expanded 124643 -> 501826 (decompressed 402.6 percent)
02:42:21:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=124643 data_size=501826, decompressed_data_size=501826 diff=0
02:42:21:WU02:FS00:0x15:- Digital signature verified
02:42:21:WU02:FS00:0x15:
02:42:21:WU02:FS00:0x15:Project: 7623 (Run 577, Clone 0, Gen 31)
02:42:21:WU02:FS00:0x15:
02:42:21:WU02:FS00:0x15:Assembly optimizations on if available.
02:42:21:WU02:FS00:0x15:Entering M.D.
02:42:23:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1256813775 3938585990 3584518017 2260536563 4266023672
02:42:23:WU02:FS00:0x15:GPU device id=0
02:42:23:WU02:FS00:0x15:Working on Protein
02:42:23:WU02:FS00:0x15:Client config unavailable.
02:42:23:WU02:FS00:0x15:Starting GUI Server
02:43:29:WU02:FS00:0x15:Setting checkpoint frequency: 400000
02:43:29:WU02:FS00:0x15:Completed         3 out of 40000000 steps (0%).
02:51:16:WU02:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
02:59:03:WU02:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
03:06:50:WU02:FS00:0x15:Completed   1200000 out of 40000000 steps (3%).
03:14:37:WU02:FS00:0x15:Completed   1600000 out of 40000000 steps (4%).
03:22:24:WU02:FS00:0x15:Completed   2000000 out of 40000000 steps (5%).
03:30:11:WU02:FS00:0x15:Completed   2400000 out of 40000000 steps (6%).
03:37:58:WU02:FS00:0x15:Completed   2800000 out of 40000000 steps (7%).
03:45:44:WU02:FS00:0x15:Completed   3200000 out of 40000000 steps (8%).
03:53:31:WU02:FS00:0x15:Completed   3600000 out of 40000000 steps (9%).
04:01:18:WU02:FS00:0x15:Completed   4000000 out of 40000000 steps (10%).
04:09:05:WU02:FS00:0x15:Completed   4400000 out of 40000000 steps (11%).
04:16:52:WU02:FS00:0x15:Completed   4800000 out of 40000000 steps (12%).
04:24:38:WU02:FS00:0x15:Completed   5200000 out of 40000000 steps (13%).
04:32:25:WU02:FS00:0x15:Completed   5600000 out of 40000000 steps (14%).
04:40:12:WU02:FS00:0x15:Completed   6000000 out of 40000000 steps (15%).
04:47:59:WU02:FS00:0x15:Completed   6400000 out of 40000000 steps (16%).
04:55:46:WU02:FS00:0x15:Completed   6800000 out of 40000000 steps (17%).
05:03:33:WU02:FS00:0x15:Completed   7200000 out of 40000000 steps (18%).
05:11:19:WU02:FS00:0x15:Completed   7600000 out of 40000000 steps (19%).
05:19:06:WU02:FS00:0x15:Completed   8000000 out of 40000000 steps (20%).
05:26:53:WU02:FS00:0x15:Completed   8400000 out of 40000000 steps (21%).
05:34:40:WU02:FS00:0x15:Completed   8800000 out of 40000000 steps (22%).
05:42:27:WU02:FS00:0x15:Completed   9200000 out of 40000000 steps (23%).
05:50:14:WU02:FS00:0x15:Completed   9600000 out of 40000000 steps (24%).
05:58:01:WU02:FS00:0x15:Completed  10000000 out of 40000000 steps (25%).
06:05:48:WU02:FS00:0x15:Completed  10400000 out of 40000000 steps (26%).
06:13:34:WU02:FS00:0x15:Completed  10800000 out of 40000000 steps (27%).
06:21:21:WU02:FS00:0x15:Completed  11200000 out of 40000000 steps (28%).
06:29:08:WU02:FS00:0x15:Completed  11600000 out of 40000000 steps (29%).
06:36:55:WU02:FS00:0x15:Completed  12000000 out of 40000000 steps (30%).
06:44:42:WU02:FS00:0x15:Completed  12400000 out of 40000000 steps (31%).
06:50:25:WU02:FS00:0x15:Run: exception thrown in GuardedRun -- cannot continue further.
06:50:25:WU02:FS00:0x15:Going to send back what have done -- stepsTotalG=40000000
06:50:25:WU02:FS00:0x15:Work fraction=0.3173 steps=40000000.
06:50:29:WU02:FS00:0x15:logfile size=19740 infoLength=19740 edr=0 trr=23
06:50:29:WU02:FS00:0x15:+ Opened results file
06:50:29:WU02:FS00:0x15:- Writing 20276 bytes of core data to disk...
06:50:29:WU02:FS00:0x15:Done: 19764 -> 5350 (compressed to 27.0 percent)
06:50:29:WU02:FS00:0x15:  ... Done.
06:50:29:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
06:50:29:WU02:FS00:0x15:
06:50:29:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
06:50:29:WARNING:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
06:50:29:WU02:FS00:Sending unit results: id:02 state:SEND error:FAULTY project:7623 run:577 clone:0 gen:31 core:0x15 unit:0x0000002e664f2dd14fe4fbaed5c603f0
06:50:29:WU02:FS00:Uploading 5.72KiB to 171.64.65.105
06:50:29:WU02:FS00:Connecting to 171.64.65.105:8080
06:50:29:WU01:FS00:Connecting to assign-GPU.stanford.edu:80
06:50:29:WU02:FS00:Upload complete
06:50:29:WU02:FS00:Server responded WORK_ACK (400)
06:50:29:WU02:FS00:Cleaning up
06:50:29:WU01:FS00:News: Welcome to Folding@Home
06:50:29:WU01:FS00:Assigned to work server 171.64.65.105
06:50:29:WU01:FS00:Requesting new work unit for slot 00: READY gpu:0:"GF114 [GeForce GTX 560 Ti]" from 171.64.65.105
06:50:29:WU01:FS00:Connecting to 171.64.65.105:8080
06:50:30:WU01:FS00:Downloading 123.51KiB
06:50:30:WU01:FS00:Download complete
06:50:30:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:7626 run:606 clone:3 gen:10 core:0x15 unit:0x0000000c664f2dd14fe61d3ddafadcaa
06:50:30:WU01:FS00:Starting
06:50:30:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 01 -suffix 01 -version 702 -lifeline 3544 -checkpoint 30 -gpu 0
06:50:30:WU01:FS00:Started FahCore on PID 3860
06:50:30:WU01:FS00:Core PID:3264
06:50:30:WU01:FS00:FahCore 0x15 started
06:50:30:WU01:FS00:Downloading project 7626 description
06:50:30:WU01:FS00:Connecting to fah-web.stanford.edu:80
06:50:30:WU01:FS00:Project 7626 description downloaded successfully
06:50:30:WU01:FS00:0x15:
06:50:30:WU01:FS00:0x15:*------------------------------*
06:50:30:WU01:FS00:0x15:Folding@Home GPU Core
06:50:30:WU01:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
06:50:30:WU01:FS00:0x15:Build host             AmoebaRemote
06:50:30:WU01:FS00:0x15:Board Type             NVIDIA/CUDA
06:50:30:WU01:FS00:0x15:Core                   15
06:50:30:WU01:FS00:0x15:
06:50:30:WU01:FS00:0x15:Window's signal control handler registered.
06:50:30:WU01:FS00:0x15:Preparing to commence simulation
06:50:30:WU01:FS00:0x15:- Looking at optimizations...
06:50:30:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
06:50:30:WU01:FS00:0x15:- Created dyn
06:50:30:WU01:FS00:0x15:- Files status OK
06:50:30:WU01:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
06:50:30:WU01:FS00:0x15:- Expanded 125960 -> 507182 (decompressed 402.6 percent)
06:50:30:WU01:FS00:0x15:Called DecompressByteArray: compressed_data_size=125960 data_size=507182, decompressed_data_size=507182 diff=0
06:50:30:WU01:FS00:0x15:- Digital signature verified
06:50:30:WU01:FS00:0x15:
06:50:30:WU01:FS00:0x15:Project: 7626 (Run 606, Clone 3, Gen 10)
06:50:30:WU01:FS00:0x15:
06:50:30:WU01:FS00:0x15:Assembly optimizations on if available.
06:50:30:WU01:FS00:0x15:Entering M.D.
06:50:32:WU01:FS00:0x15:Tpr hash 01/wudata_01.tpr:  1906049202 791482697 1629029246 970129168 3788265184
06:50:32:WU01:FS00:0x15:GPU device id=0
06:50:32:WU01:FS00:0x15:Working on Protein
06:50:32:WU01:FS00:0x15:Client config unavailable.
06:50:32:WU01:FS00:0x15:Starting GUI Server
06:51:38:WU01:FS00:0x15:Setting checkpoint frequency: 400000
06:51:38:WU01:FS00:0x15:Completed         3 out of 40000000 steps (0%).
06:59:43:WU01:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
07:07:49:WU01:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
07:15:54:WU01:FS00:0x15:Completed   1200000 out of 40000000 steps (3%).
07:24:00:WU01:FS00:0x15:Completed   1600000 out of 40000000 steps (4%).
07:32:06:WU01:FS00:0x15:Completed   2000000 out of 40000000 steps (5%).
07:40:11:WU01:FS00:0x15:Completed   2400000 out of 40000000 steps (6%).
******************************** Date: 07/01/13 ********************************
07:48:17:WU01:FS00:0x15:Completed   2800000 out of 40000000 steps (7%).
07:56:22:WU01:FS00:0x15:Completed   3200000 out of 40000000 steps (8%).
08:04:28:WU01:FS00:0x15:Completed   3600000 out of 40000000 steps (9%).
08:04:56:WU01:FS00:0x15:Run: exception thrown in GuardedRun -- cannot continue further.
08:04:56:WU01:FS00:0x15:Going to send back what have done -- stepsTotalG=40000000
08:04:56:WU01:FS00:0x15:Work fraction=0.0906 steps=40000000.
08:05:00:WU01:FS00:0x15:logfile size=15326 infoLength=15326 edr=0 trr=23
08:05:00:WU01:FS00:0x15:+ Opened results file
08:05:00:WU01:FS00:0x15:- Writing 15862 bytes of core data to disk...
08:05:00:WU01:FS00:0x15:Done: 15350 -> 4874 (compressed to 31.7 percent)
08:05:00:WU01:FS00:0x15:  ... Done.
08:05:00:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
08:05:00:WU01:FS00:0x15:
08:05:00:WU01:FS00:0x15:Folding@home Core Shutdown: EARLY_UNIT_END
08:05:01:WARNING:WU01:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
08:05:01:WU01:FS00:Sending unit results: id:01 state:SEND error:FAULTY project:7626 run:606 clone:3 gen:10 core:0x15 unit:0x0000000c664f2dd14fe61d3ddafadcaa
08:05:01:WU01:FS00:Uploading 5.26KiB to 171.64.65.105
08:05:01:WU01:FS00:Connecting to 171.64.65.105:8080
08:05:01:WU01:FS00:Upload complete
08:05:01:WU01:FS00:Server responded WORK_ACK (400)
08:05:01:WU01:FS00:Cleaning up
08:05:01:WU00:FS00:Connecting to assign-GPU.stanford.edu:80
08:05:01:WU00:FS00:News: Welcome to Folding@Home
08:05:01:WU00:FS00:Assigned to work server 171.64.65.105
08:05:01:WU00:FS00:Requesting new work unit for slot 00: READY gpu:0:"GF114 [GeForce GTX 560 Ti]" from 171.64.65.105
08:05:01:WU00:FS00:Connecting to 171.64.65.105:8080
08:05:02:WU00:FS00:Downloading 122.09KiB
08:05:02:WU00:FS00:Download complete
08:05:02:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:7623 run:388 clone:3 gen:8 core:0x15 unit:0x0000000b664f2dd14fe4faa2720b4c13
08:05:02:WU00:FS00:Starting
08:05:02:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 702 -lifeline 3544 -checkpoint 30 -gpu 0
08:05:02:WU00:FS00:Started FahCore on PID 3944
08:05:02:WU00:FS00:Core PID:3668
08:05:02:WU00:FS00:FahCore 0x15 started
08:05:02:WU00:FS00:0x15:
08:05:02:WU00:FS00:0x15:*------------------------------*
08:05:02:WU00:FS00:0x15:Folding@Home GPU Core
08:05:02:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
08:05:02:WU00:FS00:0x15:Build host             AmoebaRemote
08:05:02:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
08:05:02:WU00:FS00:0x15:Core                   15
08:05:02:WU00:FS00:0x15:
08:05:02:WU00:FS00:0x15:Window's signal control handler registered.
08:05:02:WU00:FS00:0x15:Preparing to commence simulation
08:05:02:WU00:FS00:0x15:- Looking at optimizations...
08:05:02:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
08:05:02:WU00:FS00:0x15:- Created dyn
08:05:02:WU00:FS00:0x15:- Files status OK
08:05:02:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
08:05:02:WU00:FS00:0x15:- Expanded 124512 -> 501826 (decompressed 403.0 percent)
08:05:02:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=124512 data_size=501826, decompressed_data_size=501826 diff=0
08:05:02:WU00:FS00:0x15:- Digital signature verified
08:05:02:WU00:FS00:0x15:
08:05:02:WU00:FS00:0x15:Project: 7623 (Run 388, Clone 3, Gen 8)
08:05:02:WU00:FS00:0x15:
08:05:02:WU00:FS00:0x15:Assembly optimizations on if available.
08:05:02:WU00:FS00:0x15:Entering M.D.
08:05:04:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  3070214153 1541183034 3714234059 897983637 3735042934
08:05:04:WU00:FS00:0x15:GPU device id=0
08:05:04:WU00:FS00:0x15:Working on Protein
08:05:04:WU00:FS00:0x15:Client config unavailable.
08:05:04:WU00:FS00:0x15:Starting GUI Server
08:06:10:WU00:FS00:0x15:Setting checkpoint frequency: 400000
08:06:10:WU00:FS00:0x15:Completed         3 out of 40000000 steps (0%).
08:13:56:WU00:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
08:21:43:WU00:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
08:29:30:WU00:FS00:0x15:Completed   1200000 out of 40000000 steps (3%).
08:37:17:WU00:FS00:0x15:Completed   1600000 out of 40000000 steps (4%).
08:45:03:WU00:FS00:0x15:Completed   2000000 out of 40000000 steps (5%).
08:52:50:WU00:FS00:0x15:Completed   2400000 out of 40000000 steps (6%).
09:00:37:WU00:FS00:0x15:Completed   2800000 out of 40000000 steps (7%).
09:08:24:WU00:FS00:0x15:Completed   3200000 out of 40000000 steps (8%).
09:16:11:WU00:FS00:0x15:Completed   3600000 out of 40000000 steps (9%).
09:23:58:WU00:FS00:0x15:Completed   4000000 out of 40000000 steps (10%).
09:31:45:WU00:FS00:0x15:Completed   4400000 out of 40000000 steps (11%).
09:39:31:WU00:FS00:0x15:Completed   4800000 out of 40000000 steps (12%).
09:47:18:WU00:FS00:0x15:Completed   5200000 out of 40000000 steps (13%).
09:55:05:WU00:FS00:0x15:Completed   5600000 out of 40000000 steps (14%).
10:02:52:WU00:FS00:0x15:Completed   6000000 out of 40000000 steps (15%).
10:10:38:WU00:FS00:0x15:Completed   6400000 out of 40000000 steps (16%).
10:18:25:WU00:FS00:0x15:Completed   6800000 out of 40000000 steps (17%).
10:26:12:WU00:FS00:0x15:Completed   7200000 out of 40000000 steps (18%).
10:33:59:WU00:FS00:0x15:Completed   7600000 out of 40000000 steps (19%).
10:41:33:WU00:FS00:0x15:Completed   8000000 out of 40000000 steps (20%).
10:41:34:WU00:FS00:0x15:mdrun_gpu returned 52
10:41:34:WU00:FS00:0x15:NANs detected on GPU
10:41:34:WU00:FS00:0x15:
10:41:34:WU00:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
10:41:34:WARNING:WU00:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
10:41:34:WU00:FS00:Starting
10:41:34:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 702 -lifeline 3544 -checkpoint 30 -gpu 0
10:41:34:WU00:FS00:Started FahCore on PID 2284
10:41:34:WU00:FS00:Core PID:3196
10:41:34:WU00:FS00:FahCore 0x15 started
10:41:34:WU00:FS00:0x15:
10:41:34:WU00:FS00:0x15:*------------------------------*
10:41:34:WU00:FS00:0x15:Folding@Home GPU Core
10:41:34:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
10:41:34:WU00:FS00:0x15:Build host             AmoebaRemote
10:41:34:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
10:41:34:WU00:FS00:0x15:Core                   15
10:41:34:WU00:FS00:0x15:
10:41:34:WU00:FS00:0x15:Window's signal control handler registered.
10:41:34:WU00:FS00:0x15:Preparing to commence simulation
10:41:34:WU00:FS00:0x15:- Looking at optimizations...
10:41:34:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
10:41:34:WU00:FS00:0x15:- Created dyn
10:41:34:WU00:FS00:0x15:- Files status OK
10:41:34:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
10:41:34:WU00:FS00:0x15:- Expanded 124512 -> 501826 (decompressed 403.0 percent)
10:41:34:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=124512 data_size=501826, decompressed_data_size=501826 diff=0
10:41:34:WU00:FS00:0x15:- Digital signature verified
10:41:34:WU00:FS00:0x15:
10:41:34:WU00:FS00:0x15:Project: 7623 (Run 388, Clone 3, Gen 8)
10:41:34:WU00:FS00:0x15:
10:41:34:WU00:FS00:0x15:Assembly optimizations on if available.
10:41:34:WU00:FS00:0x15:Entering M.D.
10:41:36:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  3070214153 1541183034 3714234059 897983637 3735042934
10:41:36:WU00:FS00:0x15:GPU device id=0
10:41:37:WU00:FS00:0x15:Working on Protein
10:41:37:WU00:FS00:0x15:Client config unavailable.
10:41:37:WU00:FS00:0x15:Starting GUI Server
10:42:42:WU00:FS00:0x15:Setting checkpoint frequency: 400000
10:42:42:WU00:FS00:0x15:Completed         3 out of 40000000 steps (0%).
10:42:43:WARNING:WU00:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
10:50:29:WU00:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
10:58:16:WU00:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
10:58:26:WU00:FS00:0x15:Run: exception thrown in GuardedRun -- cannot continue further.
10:58:26:WU00:FS00:0x15:Going to send back what have done -- stepsTotalG=40000000
10:58:26:WU00:FS00:0x15:Work fraction=0.0202 steps=40000000.
10:58:31:WU00:FS00:0x15:logfile size=14364 infoLength=14364 edr=0 trr=23
10:58:31:WU00:FS00:0x15:+ Opened results file
10:58:31:WU00:FS00:0x15:- Writing 14900 bytes of core data to disk...
10:58:31:WU00:FS00:0x15:Done: 14388 -> 4698 (compressed to 32.6 percent)
10:58:31:WU00:FS00:0x15:  ... Done.
10:58:31:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
10:58:31:WU00:FS00:0x15:
10:58:31:WU00:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
10:58:31:WARNING:WU00:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
10:58:31:WU00:FS00:Sending unit results: id:00 state:SEND error:FAULTY project:7623 run:388 clone:3 gen:8 core:0x15 unit:0x0000000b664f2dd14fe4faa2720b4c13
10:58:31:WU00:FS00:Uploading 5.09KiB to 171.64.65.105
10:58:31:WU00:FS00:Connecting to 171.64.65.105:8080
10:58:31:WU00:FS00:Upload complete
10:58:31:WU00:FS00:Server responded WORK_ACK (400)
10:58:31:WU00:FS00:Cleaning up
10:58:31:WU01:FS00:Connecting to assign-GPU.stanford.edu:80
10:58:32:WU01:FS00:News: Welcome to Folding@Home
10:58:32:WU01:FS00:Assigned to work server 171.64.65.105
10:58:32:WU01:FS00:Requesting new work unit for slot 00: READY gpu:0:"GF114 [GeForce GTX 560 Ti]" from 171.64.65.105
10:58:32:WU01:FS00:Connecting to 171.64.65.105:8080
10:58:32:WU01:FS00:Downloading 114.49KiB
10:58:32:WU01:FS00:Download complete
10:58:32:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:7625 run:344 clone:4 gen:0 core:0x15 unit:0x00000000664f2dd14fe6171d70721c44
10:58:32:WU01:FS00:Starting
10:58:32:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 01 -suffix 01 -version 702 -lifeline 3544 -checkpoint 30 -gpu 0
10:58:32:WU01:FS00:Started FahCore on PID 876
10:58:32:WU01:FS00:Core PID:2276
10:58:32:WU01:FS00:FahCore 0x15 started
10:58:32:WU01:FS00:Downloading project 7625 description
10:58:32:WU01:FS00:Connecting to fah-web.stanford.edu:80
10:58:33:WU01:FS00:Project 7625 description downloaded successfully
10:58:33:WU01:FS00:0x15:
10:58:33:WU01:FS00:0x15:*------------------------------*
10:58:33:WU01:FS00:0x15:Folding@Home GPU Core
10:58:33:WU01:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
10:58:33:WU01:FS00:0x15:Build host             AmoebaRemote
10:58:33:WU01:FS00:0x15:Board Type             NVIDIA/CUDA
10:58:33:WU01:FS00:0x15:Core                   15
10:58:33:WU01:FS00:0x15:
10:58:33:WU01:FS00:0x15:Window's signal control handler registered.
10:58:33:WU01:FS00:0x15:Preparing to commence simulation
10:58:33:WU01:FS00:0x15:- Looking at optimizations...
10:58:33:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
10:58:33:WU01:FS00:0x15:- Created dyn
10:58:33:WU01:FS00:0x15:- Files status OK
10:58:33:WU01:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
10:58:33:WU01:FS00:0x15:- Expanded 116724 -> 502918 (decompressed 430.8 percent)
10:58:33:WU01:FS00:0x15:Called DecompressByteArray: compressed_data_size=116724 data_size=502918, decompressed_data_size=502918 diff=0
10:58:33:WU01:FS00:0x15:- Digital signature verified
10:58:33:WU01:FS00:0x15:
10:58:33:WU01:FS00:0x15:Project: 7625 (Run 344, Clone 4, Gen 0)
10:58:33:WU01:FS00:0x15:
10:58:33:WU01:FS00:0x15:Assembly optimizations on if available.
10:58:33:WU01:FS00:0x15:Entering M.D.
10:58:34:WU01:FS00:0x15:Tpr hash 01/wudata_01.tpr:  4042329345 3972379472 918165203 3984458970 2385982189
10:58:34:WU01:FS00:0x15:GPU device id=0
10:58:34:WU01:FS00:0x15:Working on Protein
10:58:34:WU01:FS00:0x15:Client config unavailable.
10:58:34:WU01:FS00:0x15:Starting GUI Server
10:59:40:WU01:FS00:0x15:Setting checkpoint frequency: 400000
10:59:40:WU01:FS00:0x15:Completed         3 out of 40000000 steps (0%).
11:07:45:WU01:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
11:15:50:WU01:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
11:23:56:WU01:FS00:0x15:Completed   1200000 out of 40000000 steps (3%).
11:32:01:WU01:FS00:0x15:Completed   1600000 out of 40000000 steps (4%).
11:40:06:WU01:FS00:0x15:Completed   2000000 out of 40000000 steps (5%).
11:48:11:WU01:FS00:0x15:Completed   2400000 out of 40000000 steps (6%).
11:49:41:WU01:FS00:0x15:Run: exception thrown in GuardedRun -- cannot continue further.
11:49:41:WU01:FS00:0x15:Going to send back what have done -- stepsTotalG=40000000
11:49:41:WU01:FS00:0x15:Work fraction=0.0619 steps=40000000.
11:49:45:WU01:FS00:0x15:logfile size=15129 infoLength=15129 edr=0 trr=23
11:49:45:WU01:FS00:0x15:+ Opened results file
11:49:45:WU01:FS00:0x15:- Writing 15665 bytes of core data to disk...
11:49:45:WU01:FS00:0x15:Done: 15153 -> 4805 (compressed to 31.7 percent)
11:49:45:WU01:FS00:0x15:  ... Done.
11:49:45:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
11:49:45:WU01:FS00:0x15:
11:49:45:WU01:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
11:49:46:WARNING:WU01:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
11:49:46:WU01:FS00:Sending unit results: id:01 state:SEND error:FAULTY project:7625 run:344 clone:4 gen:0 core:0x15 unit:0x00000000664f2dd14fe6171d70721c44
11:49:46:WU01:FS00:Uploading 5.19KiB to 171.64.65.105
11:49:46:WU01:FS00:Connecting to 171.64.65.105:8080
11:49:46:WU01:FS00:Upload complete
11:49:46:WU01:FS00:Server responded WORK_ACK (400)
11:49:46:WU01:FS00:Cleaning up
******************************** Date: 07/01/13 ********************************

No overclock, and the card is fairly well ventilated. Temperatures are staying around 140F when running project 7623.

[GreyWhiskers]

Well, it looks like the unbroken run of 762x has come to an end. From Dec 19th until late this afternoon (Jan 8 2013 PST) I had gotten nothing but over 100 of these on my three Fermi/Kepler GPUs. Just when I brought my big desktop back up after installing my GTX560Ti that came back from MSI as an RMA replacement for the Twin Frozr fans, my other three folders all started loading P8073s. The newly reinstalled GTX560Ti also picked up a P8073 when it came back up.

FYI, I have GT430; GTX560M (laptop); GTX560Ti; and GTX660Ti all processing P8073s.

The P8073s have a lower ppd on all of the hardware than the P762x WUs.

[art_l_j_PlanetAMD64]

Well, it looks like the unbroken run of 762x has come to an end.
The P8073s have a lower ppd on all of the hardware than the P762x WUs.

That's too bad, the 762x WUs gave great PPDs. My rig with the dual GTX460s must have picked up a couple of the last few 762x WUs left, it got a pair of 7623s about an hour ago.

[Jesse_V]

More errors with these projects:

Code: Select all

(note, I snipped the log: 7626 (255, 3, 21) failed a number of other times at different frames but with the same error)
12:34:43:WU00:FS00:0x15:*------------------------------*
12:34:43:WU00:FS00:0x15:Folding@Home GPU Core
12:34:43:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
12:34:43:WU00:FS00:0x15:Build host             AmoebaRemote
12:34:43:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
12:34:43:WU00:FS00:0x15:Core                   15
12:34:43:WU00:FS00:0x15:
12:34:43:WU00:FS00:0x15:Window's signal control handler registered.
12:34:43:WU00:FS00:0x15:Preparing to commence simulation
12:34:43:WU00:FS00:0x15:- Looking at optimizations...
12:34:43:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
12:34:43:WU00:FS00:0x15:- Created dyn
12:34:43:WU00:FS00:0x15:- Files status OK
12:34:43:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
12:34:43:WU00:FS00:0x15:- Expanded 126153 -> 507182 (decompressed 402.0 percent)
12:34:43:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=126153 data_size=507182, decompressed_data_size=507182 diff=0
12:34:43:WU00:FS00:0x15:- Digital signature verified
12:34:43:WU00:FS00:0x15:
12:34:43:WU00:FS00:0x15:Project: 7626 (Run 355, Clone 3, Gen 21)
12:34:43:WU00:FS00:0x15:
12:34:43:WU00:FS00:0x15:Assembly optimizations on if available.
12:34:43:WU00:FS00:0x15:Entering M.D.
12:34:45:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  370896658 420092387 4056984041 2729292146 1452050292
12:34:45:WU00:FS00:0x15:GPU device id=0
12:34:45:WU00:FS00:0x15:Working on Protein
12:34:45:WU00:FS00:0x15:Client config unavailable.
12:34:45:WU00:FS00:0x15:Starting GUI Server
12:35:51:WU00:FS00:0x15:Setting checkpoint frequency: 400000
12:35:51:WU00:FS00:0x15:Completed         3 out of 40000000 steps (0%).
12:35:52:WARNING:WU00:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
12:43:56:WU00:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
(snip)
13:39:34:WU00:FS00:0x15:Completed   3200000 out of 40000000 steps (8%).
13:39:34:WU00:FS00:0x15:mdrun_gpu returned 52
13:39:34:WU00:FS00:0x15:NANs detected on GPU
13:39:34:WU00:FS00:0x15:
13:39:34:WU00:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
13:39:35:WARNING:WU00:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
13:39:35:WARNING:WU00:FS00:Too many errors, failing
13:39:35:WU00:FS00:Sending unit results: id:00 state:SEND error:FAILED project:7626 run:355 clone:3 gen:21 core:0x15 unit:0x00000017664f2dd14fe61bcb646145d9
13:39:35:WU00:FS00:Connecting to 171.64.65.105:8080
13:39:35:WU00:FS00:Server responded WORK_QUIT (404)
13:39:35:WARNING:WU00:FS00:Server did not like results, dumping
13:39:35:WU00:FS00:Cleaning up
13:39:35:WU02:FS00:Connecting to assign-GPU.stanford.edu:80
13:39:36:WU02:FS00:News: Welcome to Folding@Home
13:39:36:WU02:FS00:Assigned to work server 171.64.65.105
13:39:36:WU02:FS00:Requesting new work unit for slot 00: READY gpu:0:GF114 [GeForce GTX 560 Ti] from 171.64.65.105
13:39:36:WU02:FS00:Connecting to 171.64.65.105:8080
13:39:36:WU02:FS00:Downloading 122.21KiB
13:39:36:WU02:FS00:Download complete
13:39:37:WU02:FS00:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:7623 run:572 clone:6 gen:14 core:0x15 unit:0x0000000e664f2dd14fe4fba8c3014280
13:39:37:WU02:FS00:Starting
13:39:37:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
13:39:37:WU02:FS00:Started FahCore on PID 4464
13:39:37:WU02:FS00:Core PID:3960
13:39:37:WU02:FS00:FahCore 0x15 started
13:39:37:WU02:FS00:0x15:
13:39:37:WU02:FS00:0x15:*------------------------------*
13:39:37:WU02:FS00:0x15:Folding@Home GPU Core
13:39:37:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:39:37:WU02:FS00:0x15:Build host             AmoebaRemote
13:39:37:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
13:39:37:WU02:FS00:0x15:Core                   15
13:39:37:WU02:FS00:0x15:
13:39:37:WU02:FS00:0x15:Window's signal control handler registered.
13:39:37:WU02:FS00:0x15:Preparing to commence simulation
13:39:37:WU02:FS00:0x15:- Looking at optimizations...
13:39:37:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
13:39:37:WU02:FS00:0x15:- Created dyn
13:39:37:WU02:FS00:0x15:- Files status OK
13:39:37:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:39:37:WU02:FS00:0x15:- Expanded 124631 -> 501826 (decompressed 402.6 percent)
13:39:37:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=124631 data_size=501826, decompressed_data_size=501826 diff=0
13:39:37:WU02:FS00:0x15:- Digital signature verified
13:39:37:WU02:FS00:0x15:
13:39:37:WU02:FS00:0x15:Project: 7623 (Run 572, Clone 6, Gen 14)
13:39:37:WU02:FS00:0x15:
13:39:37:WU02:FS00:0x15:Assembly optimizations on if available.
13:39:37:WU02:FS00:0x15:Entering M.D.
13:39:39:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1509011994 16233882 3118172587 658764004 634297493
13:39:39:WU02:FS00:0x15:GPU device id=0
13:39:39:WU02:FS00:0x15:Working on Protein
13:39:39:WU02:FS00:0x15:Client config unavailable.
13:39:39:WU02:FS00:0x15:Starting GUI Server
13:40:45:WU02:FS00:0x15:Setting checkpoint frequency: 400000
13:40:45:WU02:FS00:0x15:Completed         3 out of 40000000 steps (0%).
13:48:07:WU02:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
13:48:07:WU02:FS00:0x15:mdrun_gpu returned 52
13:48:07:WU02:FS00:0x15:NANs detected on GPU
13:48:07:WU02:FS00:0x15:
13:48:07:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
13:48:07:WARNING:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
13:48:08:WU02:FS00:Starting
13:48:08:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
13:48:08:WU02:FS00:Started FahCore on PID 792
13:48:08:WU02:FS00:Core PID:2400
13:48:08:WU02:FS00:FahCore 0x15 started
13:48:08:WU02:FS00:0x15:
13:48:08:WU02:FS00:0x15:*------------------------------*
13:48:08:WU02:FS00:0x15:Folding@Home GPU Core
13:48:08:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:48:08:WU02:FS00:0x15:Build host             AmoebaRemote
13:48:08:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
13:48:08:WU02:FS00:0x15:Core                   15
13:48:08:WU02:FS00:0x15:
13:48:08:WU02:FS00:0x15:Window's signal control handler registered.
13:48:08:WU02:FS00:0x15:Preparing to commence simulation
13:48:08:WU02:FS00:0x15:- Looking at optimizations...
13:48:08:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
13:48:08:WU02:FS00:0x15:- Created dyn
13:48:08:WU02:FS00:0x15:- Files status OK
13:48:08:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:48:08:WU02:FS00:0x15:- Expanded 124631 -> 501826 (decompressed 402.6 percent)
13:48:08:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=124631 data_size=501826, decompressed_data_size=501826 diff=0
13:48:08:WU02:FS00:0x15:- Digital signature verified
13:48:08:WU02:FS00:0x15:
13:48:08:WU02:FS00:0x15:Project: 7623 (Run 572, Clone 6, Gen 14)
13:48:08:WU02:FS00:0x15:
13:48:08:WU02:FS00:0x15:Assembly optimizations on if available.
13:48:08:WU02:FS00:0x15:Entering M.D.
13:48:10:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1509011994 16233882 3118172587 658764004 634297493
13:48:10:WU02:FS00:0x15:GPU device id=0
13:48:10:WU02:FS00:0x15:Working on Protein
13:48:10:WU02:FS00:0x15:Client config unavailable.
13:48:10:WU02:FS00:0x15:Starting GUI Server
13:49:16:WU02:FS00:0x15:Setting checkpoint frequency: 400000
13:49:16:WU02:FS00:0x15:Completed         3 out of 40000000 steps (0%).
13:49:17:WARNING:WU02:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
13:56:40:WU02:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
13:56:41:WU02:FS00:0x15:mdrun_gpu returned 52
13:56:41:WU02:FS00:0x15:NANs detected on GPU
13:56:41:WU02:FS00:0x15:
13:56:41:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
13:56:41:WARNING:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
13:56:41:WU02:FS00:Starting
13:56:41:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
13:56:41:WU02:FS00:Started FahCore on PID 4092
13:56:41:WU02:FS00:Core PID:4060
13:56:41:WU02:FS00:FahCore 0x15 started
13:56:42:WU02:FS00:0x15:
13:56:42:WU02:FS00:0x15:*------------------------------*
13:56:42:WU02:FS00:0x15:Folding@Home GPU Core
13:56:42:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:56:42:WU02:FS00:0x15:Build host             AmoebaRemote
13:56:42:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
13:56:42:WU02:FS00:0x15:Core                   15
13:56:42:WU02:FS00:0x15:
13:56:42:WU02:FS00:0x15:Window's signal control handler registered.
13:56:42:WU02:FS00:0x15:Preparing to commence simulation
13:56:42:WU02:FS00:0x15:- Looking at optimizations...
13:56:42:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
13:56:42:WU02:FS00:0x15:- Created dyn
13:56:42:WU02:FS00:0x15:- Files status OK
13:56:42:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:56:42:WU02:FS00:0x15:- Expanded 124631 -> 501826 (decompressed 402.6 percent)
13:56:42:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=124631 data_size=501826, decompressed_data_size=501826 diff=0
13:56:42:WU02:FS00:0x15:- Digital signature verified
13:56:42:WU02:FS00:0x15:
13:56:42:WU02:FS00:0x15:Project: 7623 (Run 572, Clone 6, Gen 14)
13:56:42:WU02:FS00:0x15:
13:56:42:WU02:FS00:0x15:Assembly optimizations on if available.
13:56:42:WU02:FS00:0x15:Entering M.D.
13:56:43:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1509011994 16233882 3118172587 658764004 634297493
13:56:43:WU02:FS00:0x15:GPU device id=0
13:56:43:WU02:FS00:0x15:Working on Protein
13:56:43:WU02:FS00:0x15:Client config unavailable.
13:56:43:WU02:FS00:0x15:Starting GUI Server
13:57:49:WU02:FS00:0x15:Setting checkpoint frequency: 400000
13:57:49:WU02:FS00:0x15:Completed         3 out of 40000000 steps (0%).
13:57:50:WARNING:WU02:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
14:05:36:WU02:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
14:12:48:WU02:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
14:12:49:WU02:FS00:0x15:mdrun_gpu returned 52
14:12:49:WU02:FS00:0x15:NANs detected on GPU
14:12:49:WU02:FS00:0x15:
14:12:49:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
14:12:49:WARNING:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
14:12:49:WU02:FS00:Starting
14:12:49:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
14:12:49:WU02:FS00:Started FahCore on PID 2116
14:12:49:WU02:FS00:Core PID:296
14:12:49:WU02:FS00:FahCore 0x15 started
14:12:50:WU02:FS00:0x15:
14:12:50:WU02:FS00:0x15:*------------------------------*
14:12:50:WU02:FS00:0x15:Folding@Home GPU Core
14:12:50:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
14:12:50:WU02:FS00:0x15:Build host             AmoebaRemote
14:12:50:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
14:12:50:WU02:FS00:0x15:Core                   15
14:12:50:WU02:FS00:0x15:
14:12:50:WU02:FS00:0x15:Window's signal control handler registered.
14:12:50:WU02:FS00:0x15:Preparing to commence simulation
14:12:50:WU02:FS00:0x15:- Looking at optimizations...
14:12:50:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
14:12:50:WU02:FS00:0x15:- Created dyn
14:12:50:WU02:FS00:0x15:- Files status OK
14:12:50:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
14:12:50:WU02:FS00:0x15:- Expanded 124631 -> 501826 (decompressed 402.6 percent)
14:12:50:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=124631 data_size=501826, decompressed_data_size=501826 diff=0
14:12:50:WU02:FS00:0x15:- Digital signature verified
14:12:50:WU02:FS00:0x15:
14:12:50:WU02:FS00:0x15:Project: 7623 (Run 572, Clone 6, Gen 14)
14:12:50:WU02:FS00:0x15:
14:12:50:WU02:FS00:0x15:Assembly optimizations on if available.
14:12:50:WU02:FS00:0x15:Entering M.D.
14:12:52:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1509011994 16233882 3118172587 658764004 634297493
14:12:52:WU02:FS00:0x15:GPU device id=0
14:12:52:WU02:FS00:0x15:Working on Protein
14:12:52:WU02:FS00:0x15:Client config unavailable.
14:12:52:WU02:FS00:0x15:Starting GUI Server
14:13:57:WU02:FS00:0x15:Setting checkpoint frequency: 400000
14:13:57:WU02:FS00:0x15:Completed         3 out of 40000000 steps (0%).
14:13:58:WARNING:WU02:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
14:21:44:WU02:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
14:28:43:WU02:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
14:28:44:WU02:FS00:0x15:mdrun_gpu returned 52
14:28:44:WU02:FS00:0x15:NANs detected on GPU
14:28:44:WU02:FS00:0x15:
14:28:44:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
14:28:44:WARNING:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
14:28:44:WU02:FS00:Starting
14:28:44:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
14:28:44:WU02:FS00:Started FahCore on PID 4200
14:28:44:WU02:FS00:Core PID:3332
14:28:44:WU02:FS00:FahCore 0x15 started
14:28:45:WU02:FS00:0x15:
14:28:45:WU02:FS00:0x15:*------------------------------*
14:28:45:WU02:FS00:0x15:Folding@Home GPU Core
14:28:45:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
14:28:45:WU02:FS00:0x15:Build host             AmoebaRemote
14:28:45:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
14:28:45:WU02:FS00:0x15:Core                   15
14:28:45:WU02:FS00:0x15:
14:28:45:WU02:FS00:0x15:Window's signal control handler registered.
14:28:45:WU02:FS00:0x15:Preparing to commence simulation
14:28:45:WU02:FS00:0x15:- Looking at optimizations...
14:28:45:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
14:28:45:WU02:FS00:0x15:- Created dyn
14:28:45:WU02:FS00:0x15:- Files status OK
14:28:45:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
14:28:45:WU02:FS00:0x15:- Expanded 124631 -> 501826 (decompressed 402.6 percent)
14:28:45:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=124631 data_size=501826, decompressed_data_size=501826 diff=0
14:28:45:WU02:FS00:0x15:- Digital signature verified
14:28:45:WU02:FS00:0x15:
14:28:45:WU02:FS00:0x15:Project: 7623 (Run 572, Clone 6, Gen 14)
14:28:45:WU02:FS00:0x15:
14:28:45:WU02:FS00:0x15:Assembly optimizations on if available.
14:28:45:WU02:FS00:0x15:Entering M.D.
14:28:47:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1509011994 16233882 3118172587 658764004 634297493
14:28:47:WU02:FS00:0x15:GPU device id=0
14:28:47:WU02:FS00:0x15:Working on Protein
14:28:47:WU02:FS00:0x15:Client config unavailable.
14:28:47:WU02:FS00:0x15:Starting GUI Server
14:29:53:WU02:FS00:0x15:Setting checkpoint frequency: 400000
14:29:53:WARNING:WU02:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
14:29:53:WU02:FS00:0x15:Completed         3 out of 40000000 steps (0%).
14:37:03:WU02:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
14:37:04:WU02:FS00:0x15:mdrun_gpu returned 52
14:37:04:WU02:FS00:0x15:NANs detected on GPU
14:37:04:WU02:FS00:0x15:
14:37:04:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
14:37:04:WARNING:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
14:37:04:WARNING:WU02:FS00:Too many errors, failing
14:37:04:WU02:FS00:Sending unit results: id:02 state:SEND error:FAILED project:7623 run:572 clone:6 gen:14 core:0x15 unit:0x0000000e664f2dd14fe4fba8c3014280
14:37:04:WU02:FS00:Connecting to 171.64.65.105:8080
14:37:05:WU02:FS00:Server responded WORK_QUIT (404)
14:37:05:WU01:FS00:Connecting to assign-GPU.stanford.edu:80
14:37:05:WARNING:WU02:FS00:Server did not like results, dumping
14:37:05:WU02:FS00:Cleaning up
14:37:05:WU01:FS00:News: Welcome to Folding@Home
14:37:05:WU01:FS00:Assigned to work server 171.64.65.105
14:37:05:WU01:FS00:Requesting new work unit for slot 00: READY gpu:0:GF114 [GeForce GTX 560 Ti] from 171.64.65.105
14:37:05:WU01:FS00:Connecting to 171.64.65.105:8080
14:37:06:WU01:FS00:Downloading 124.13KiB
14:37:06:WU01:FS00:Download complete
14:37:06:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:7624 run:192 clone:4 gen:8 core:0x15 unit:0x00000008664f2dd14fe611a2f2141b0d
14:37:06:WU01:FS00:Starting
14:37:06:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 01 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
14:37:06:WU01:FS00:Started FahCore on PID 3356
14:37:06:WU01:FS00:Core PID:2800
14:37:06:WU01:FS00:FahCore 0x15 started
14:37:06:WU01:FS00:Downloading project 7624 description
14:37:06:WU01:FS00:Connecting to fah-web.stanford.edu:80
14:37:07:WU01:FS00:0x15:
14:37:07:WU01:FS00:0x15:*------------------------------*
14:37:07:WU01:FS00:0x15:Folding@Home GPU Core
14:37:07:WU01:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
14:37:07:WU01:FS00:0x15:Build host             AmoebaRemote
14:37:07:WU01:FS00:0x15:Board Type             NVIDIA/CUDA
14:37:07:WU01:FS00:0x15:Core                   15
14:37:07:WU01:FS00:0x15:
14:37:07:WU01:FS00:0x15:Window's signal control handler registered.
14:37:07:WU01:FS00:0x15:Preparing to commence simulation
14:37:07:WU01:FS00:0x15:- Looking at optimizations...
14:37:07:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
14:37:07:WU01:FS00:0x15:- Created dyn
14:37:07:WU01:FS00:0x15:- Files status OK
14:37:07:WU01:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
14:37:07:WU01:FS00:0x15:- Expanded 126592 -> 507182 (decompressed 400.6 percent)
14:37:07:WU01:FS00:0x15:Called DecompressByteArray: compressed_data_size=126592 data_size=507182, decompressed_data_size=507182 diff=0
14:37:07:WU01:FS00:0x15:- Digital signature verified
14:37:07:WU01:FS00:0x15:
14:37:07:WU01:FS00:0x15:Project: 7624 (Run 192, Clone 4, Gen 8)
14:37:07:WU01:FS00:0x15:
14:37:07:WU01:FS00:0x15:Assembly optimizations on if available.
14:37:07:WU01:FS00:0x15:Entering M.D.
14:37:07:WU01:FS00:Project 7624 description downloaded successfully
14:37:08:WU01:FS00:0x15:Tpr hash 01/wudata_01.tpr:  2748970219 1921943655 4050475119 3784124478 3400413228
14:37:08:WU01:FS00:0x15:GPU device id=0
14:37:08:WU01:FS00:0x15:Working on Protein
14:37:08:WU01:FS00:0x15:Client config unavailable.
14:37:08:WU01:FS00:0x15:Starting GUI Server
14:38:14:WU01:FS00:0x15:Setting checkpoint frequency: 400000
14:38:14:WU01:FS00:0x15:Completed         3 out of 40000000 steps (0%).
14:46:19:WU01:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
14:53:20:WU01:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
14:53:20:WARNING:WU01:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
14:53:20:WU01:FS00:Starting
14:53:20:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 01 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
14:53:20:WU01:FS00:Started FahCore on PID 4244
14:53:20:WU01:FS00:Core PID:1932
14:53:20:WU01:FS00:FahCore 0x15 started
14:53:21:WU01:FS00:0x15:
14:53:21:WU01:FS00:0x15:*------------------------------*
14:53:21:WU01:FS00:0x15:Folding@Home GPU Core
14:53:21:WU01:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
14:53:21:WU01:FS00:0x15:Build host             AmoebaRemote
14:53:21:WU01:FS00:0x15:Board Type             NVIDIA/CUDA
14:53:21:WU01:FS00:0x15:Core                   15
14:53:21:WU01:FS00:0x15:
14:53:21:WU01:FS00:0x15:Window's signal control handler registered.
14:53:21:WU01:FS00:0x15:Preparing to commence simulation
14:53:21:WU01:FS00:0x15:- Looking at optimizations...
14:53:21:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
14:53:21:WU01:FS00:0x15:- Created dyn
14:53:21:WU01:FS00:0x15:- Files status OK
14:53:21:WU01:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
14:53:21:WU01:FS00:0x15:- Expanded 126592 -> 507182 (decompressed 400.6 percent)
14:53:21:WU01:FS00:0x15:Called DecompressByteArray: compressed_data_size=126592 data_size=507182, decompressed_data_size=507182 diff=0
14:53:21:WU01:FS00:0x15:- Digital signature verified
14:53:21:WU01:FS00:0x15:
14:53:21:WU01:FS00:0x15:Project: 7624 (Run 192, Clone 4, Gen 8)
14:53:21:WU01:FS00:0x15:
14:53:21:WU01:FS00:0x15:Assembly optimizations on if available.
14:53:21:WU01:FS00:0x15:Entering M.D.
14:53:23:WU01:FS00:0x15:Tpr hash 01/wudata_01.tpr:  2748970219 1921943655 4050475119 3784124478 3400413228
14:53:23:WU01:FS00:0x15:GPU device id=0
14:53:23:WU01:FS00:0x15:Working on Protein
14:53:23:WU01:FS00:0x15:Client config unavailable.
14:53:23:WU01:FS00:0x15:Starting GUI Server
14:54:29:WU01:FS00:0x15:Setting checkpoint frequency: 400000
14:54:29:WARNING:WU01:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
14:54:29:WU01:FS00:0x15:Completed         3 out of 40000000 steps (0%).
15:02:34:WU01:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
15:09:45:WU01:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
15:09:45:WU01:FS00:0x15:mdrun_gpu returned 52
15:09:45:WU01:FS00:0x15:NANs detected on GPU
15:09:45:WU01:FS00:0x15:
15:09:45:WU01:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
15:09:45:WARNING:WU01:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
15:09:46:WU01:FS00:Starting
15:09:46:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 01 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
15:09:46:WU01:FS00:Started FahCore on PID 932
15:09:46:WU01:FS00:Core PID:4368
15:09:46:WU01:FS00:FahCore 0x15 started
15:09:46:WU01:FS00:0x15:
15:09:46:WU01:FS00:0x15:*------------------------------*
15:09:46:WU01:FS00:0x15:Folding@Home GPU Core
15:09:46:WU01:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
15:09:46:WU01:FS00:0x15:Build host             AmoebaRemote
15:09:46:WU01:FS00:0x15:Board Type             NVIDIA/CUDA
15:09:46:WU01:FS00:0x15:Core                   15
15:09:46:WU01:FS00:0x15:
15:09:46:WU01:FS00:0x15:Window's signal control handler registered.
15:09:46:WU01:FS00:0x15:Preparing to commence simulation
15:09:46:WU01:FS00:0x15:- Looking at optimizations...
15:09:46:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
15:09:46:WU01:FS00:0x15:- Created dyn
15:09:46:WU01:FS00:0x15:- Files status OK
15:09:46:WU01:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
15:09:46:WU01:FS00:0x15:- Expanded 126592 -> 507182 (decompressed 400.6 percent)
15:09:46:WU01:FS00:0x15:Called DecompressByteArray: compressed_data_size=126592 data_size=507182, decompressed_data_size=507182 diff=0
15:09:46:WU01:FS00:0x15:- Digital signature verified
15:09:46:WU01:FS00:0x15:
15:09:46:WU01:FS00:0x15:Project: 7624 (Run 192, Clone 4, Gen 8)
15:09:46:WU01:FS00:0x15:
15:09:46:WU01:FS00:0x15:Assembly optimizations on if available.
15:09:46:WU01:FS00:0x15:Entering M.D.
15:09:48:WU01:FS00:0x15:Tpr hash 01/wudata_01.tpr:  2748970219 1921943655 4050475119 3784124478 3400413228
15:09:48:WU01:FS00:0x15:GPU device id=0
15:09:48:WU01:FS00:0x15:Working on Protein
15:09:48:WU01:FS00:0x15:Client config unavailable.
15:09:48:WU01:FS00:0x15:Starting GUI Server
15:10:54:WU01:FS00:0x15:Setting checkpoint frequency: 400000
15:10:54:WU01:FS00:0x15:Completed         3 out of 40000000 steps (0%).
15:10:55:WARNING:WU01:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
15:18:59:WU01:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
15:26:00:WU01:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
15:26:01:WU01:FS00:0x15:mdrun_gpu returned 52
15:26:01:WU01:FS00:0x15:NANs detected on GPU
15:26:01:WU01:FS00:0x15:
15:26:01:WU01:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
15:26:01:WARNING:WU01:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
15:26:01:WU01:FS00:Starting
15:26:01:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 01 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
15:26:01:WU01:FS00:Started FahCore on PID 2224
15:26:01:WU01:FS00:Core PID:2116
15:26:01:WU01:FS00:FahCore 0x15 started
15:26:02:WU01:FS00:0x15:
15:26:02:WU01:FS00:0x15:*------------------------------*
15:26:02:WU01:FS00:0x15:Folding@Home GPU Core
15:26:02:WU01:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
15:26:02:WU01:FS00:0x15:Build host             AmoebaRemote
15:26:02:WU01:FS00:0x15:Board Type             NVIDIA/CUDA
15:26:02:WU01:FS00:0x15:Core                   15
15:26:02:WU01:FS00:0x15:
15:26:02:WU01:FS00:0x15:Window's signal control handler registered.
15:26:02:WU01:FS00:0x15:Preparing to commence simulation
15:26:02:WU01:FS00:0x15:- Looking at optimizations...
15:26:02:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
15:26:02:WU01:FS00:0x15:- Created dyn
15:26:02:WU01:FS00:0x15:- Files status OK
15:26:02:WU01:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
15:26:02:WU01:FS00:0x15:- Expanded 126592 -> 507182 (decompressed 400.6 percent)
15:26:02:WU01:FS00:0x15:Called DecompressByteArray: compressed_data_size=126592 data_size=507182, decompressed_data_size=507182 diff=0
15:26:02:WU01:FS00:0x15:- Digital signature verified
15:26:02:WU01:FS00:0x15:
15:26:02:WU01:FS00:0x15:Project: 7624 (Run 192, Clone 4, Gen 8)
15:26:02:WU01:FS00:0x15:
15:26:02:WU01:FS00:0x15:Assembly optimizations on if available.
15:26:02:WU01:FS00:0x15:Entering M.D.
15:26:03:WU01:FS00:0x15:Tpr hash 01/wudata_01.tpr:  2748970219 1921943655 4050475119 3784124478 3400413228
15:26:03:WU01:FS00:0x15:GPU device id=0
15:26:03:WU01:FS00:0x15:Working on Protein
15:26:03:WU01:FS00:0x15:Client config unavailable.
15:26:04:WU01:FS00:0x15:Starting GUI Server
15:27:10:WU01:FS00:0x15:Setting checkpoint frequency: 400000
15:27:10:WARNING:WU01:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
15:27:10:WU01:FS00:0x15:Completed         3 out of 40000000 steps (0%).
15:35:14:WU01:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
15:42:18:WU01:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
15:42:19:WU01:FS00:0x15:mdrun_gpu returned 52
15:42:19:WU01:FS00:0x15:NANs detected on GPU
15:42:19:WU01:FS00:0x15:
15:42:19:WU01:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
15:42:20:WARNING:WU01:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
15:42:20:WU01:FS00:Starting
15:42:20:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 01 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
15:42:20:WU01:FS00:Started FahCore on PID 4120
15:42:20:WU01:FS00:Core PID:1632
15:42:20:WU01:FS00:FahCore 0x15 started
15:42:20:WU01:FS00:0x15:
15:42:20:WU01:FS00:0x15:*------------------------------*
15:42:20:WU01:FS00:0x15:Folding@Home GPU Core
15:42:20:WU01:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
15:42:20:WU01:FS00:0x15:Build host             AmoebaRemote
15:42:20:WU01:FS00:0x15:Board Type             NVIDIA/CUDA
15:42:20:WU01:FS00:0x15:Core                   15
15:42:20:WU01:FS00:0x15:
15:42:20:WU01:FS00:0x15:Window's signal control handler registered.
15:42:20:WU01:FS00:0x15:Preparing to commence simulation
15:42:20:WU01:FS00:0x15:- Looking at optimizations...
15:42:20:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
15:42:20:WU01:FS00:0x15:- Created dyn
15:42:20:WU01:FS00:0x15:- Files status OK
15:42:20:WU01:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
15:42:20:WU01:FS00:0x15:- Expanded 126592 -> 507182 (decompressed 400.6 percent)
15:42:20:WU01:FS00:0x15:Called DecompressByteArray: compressed_data_size=126592 data_size=507182, decompressed_data_size=507182 diff=0
15:42:20:WU01:FS00:0x15:- Digital signature verified
15:42:20:WU01:FS00:0x15:
15:42:20:WU01:FS00:0x15:Project: 7624 (Run 192, Clone 4, Gen 8)
15:42:20:WU01:FS00:0x15:
15:42:20:WU01:FS00:0x15:Assembly optimizations on if available.
15:42:20:WU01:FS00:0x15:Entering M.D.
15:42:22:WU01:FS00:0x15:Tpr hash 01/wudata_01.tpr:  2748970219 1921943655 4050475119 3784124478 3400413228
15:42:22:WU01:FS00:0x15:GPU device id=0
15:42:22:WU01:FS00:0x15:Working on Protein
15:42:22:WU01:FS00:0x15:Client config unavailable.
15:42:22:WU01:FS00:0x15:Starting GUI Server
15:43:28:WU01:FS00:0x15:Setting checkpoint frequency: 400000
15:43:28:WU01:FS00:0x15:Completed         3 out of 40000000 steps (0%).
15:43:29:WARNING:WU01:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
15:50:30:WU01:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
15:50:31:WU01:FS00:0x15:mdrun_gpu returned 52
15:50:31:WU01:FS00:0x15:NANs detected on GPU
15:50:31:WU01:FS00:0x15:
15:50:31:WU01:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
15:50:31:WARNING:WU01:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
15:50:31:WARNING:WU01:FS00:Too many errors, failing
15:50:31:WU01:FS00:Sending unit results: id:01 state:SEND error:FAILED project:7624 run:192 clone:4 gen:8 core:0x15 unit:0x00000008664f2dd14fe611a2f2141b0d
15:50:31:WU01:FS00:Connecting to 171.64.65.105:8080
15:50:31:WU02:FS00:Connecting to assign-GPU.stanford.edu:80
15:50:31:WU01:FS00:Server responded WORK_QUIT (404)
15:50:31:WARNING:WU01:FS00:Server did not like results, dumping
15:50:31:WU01:FS00:Cleaning up
15:50:32:WU02:FS00:News: Welcome to Folding@Home
15:50:32:WU02:FS00:Assigned to work server 171.64.65.105
15:50:32:WU02:FS00:Requesting new work unit for slot 00: READY gpu:0:GF114 [GeForce GTX 560 Ti] from 171.64.65.105
15:50:32:WU02:FS00:Connecting to 171.64.65.105:8080
15:50:32:WU02:FS00:Downloading 123.65KiB
15:50:32:WU02:FS00:Download complete
15:50:32:WU02:FS00:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:7624 run:684 clone:0 gen:39 core:0x15 unit:0x00000030664f2dd14fe61478c872b8ae
15:50:32:WU02:FS00:Starting
15:50:32:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
15:50:32:WU02:FS00:Started FahCore on PID 4632
15:50:32:WU02:FS00:Core PID:248
15:50:32:WU02:FS00:FahCore 0x15 started
15:50:33:WU02:FS00:0x15:
15:50:33:WU02:FS00:0x15:*------------------------------*
15:50:33:WU02:FS00:0x15:Folding@Home GPU Core
15:50:33:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
15:50:33:WU02:FS00:0x15:Build host             AmoebaRemote
15:50:33:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
15:50:33:WU02:FS00:0x15:Core                   15
15:50:33:WU02:FS00:0x15:
15:50:33:WU02:FS00:0x15:Window's signal control handler registered.
15:50:33:WU02:FS00:0x15:Preparing to commence simulation
15:50:33:WU02:FS00:0x15:- Looking at optimizations...
15:50:33:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
15:50:33:WU02:FS00:0x15:- Created dyn
15:50:33:WU02:FS00:0x15:- Files status OK
15:50:33:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
15:50:33:WU02:FS00:0x15:- Expanded 126109 -> 507182 (decompressed 402.1 percent)
15:50:33:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=126109 data_size=507182, decompressed_data_size=507182 diff=0
15:50:33:WU02:FS00:0x15:- Digital signature verified
15:50:33:WU02:FS00:0x15:
15:50:33:WU02:FS00:0x15:Project: 7624 (Run 684, Clone 0, Gen 39)
15:50:33:WU02:FS00:0x15:
15:50:33:WU02:FS00:0x15:Assembly optimizations on if available.
15:50:33:WU02:FS00:0x15:Entering M.D.
15:50:35:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  2263893614 573565463 1194305807 1390631238 1763632399
15:50:35:WU02:FS00:0x15:GPU device id=0
15:50:35:WU02:FS00:0x15:Working on Protein
15:50:35:WU02:FS00:0x15:Client config unavailable.
15:50:35:WU02:FS00:0x15:Starting GUI Server
15:51:41:WU02:FS00:0x15:Setting checkpoint frequency: 400000
15:51:41:WU02:FS00:0x15:Completed         3 out of 40000000 steps (0%).
15:59:46:WU02:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
16:07:51:WU02:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
16:15:30:WU02:FS00:0x15:Completed   1200000 out of 40000000 steps (3%).
16:15:31:WU02:FS00:0x15:mdrun_gpu returned 52
16:15:31:WU02:FS00:0x15:NANs detected on GPU
16:15:31:WU02:FS00:0x15:
16:15:31:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
16:15:31:WARNING:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
16:15:31:WU02:FS00:Starting
16:15:31:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
16:15:31:WU02:FS00:Started FahCore on PID 1364
16:15:31:WU02:FS00:Core PID:3280
16:15:31:WU02:FS00:FahCore 0x15 started
16:15:32:WU02:FS00:0x15:
16:15:32:WU02:FS00:0x15:*------------------------------*
16:15:32:WU02:FS00:0x15:Folding@Home GPU Core
16:15:32:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
16:15:32:WU02:FS00:0x15:Build host             AmoebaRemote
16:15:32:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
16:15:32:WU02:FS00:0x15:Core                   15
16:15:32:WU02:FS00:0x15:
16:15:32:WU02:FS00:0x15:Window's signal control handler registered.
16:15:32:WU02:FS00:0x15:Preparing to commence simulation
16:15:32:WU02:FS00:0x15:- Looking at optimizations...
16:15:32:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
16:15:32:WU02:FS00:0x15:- Created dyn
16:15:32:WU02:FS00:0x15:- Files status OK
16:15:32:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
16:15:32:WU02:FS00:0x15:- Expanded 126109 -> 507182 (decompressed 402.1 percent)
16:15:32:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=126109 data_size=507182, decompressed_data_size=507182 diff=0
16:15:32:WU02:FS00:0x15:- Digital signature verified
16:15:32:WU02:FS00:0x15:
16:15:32:WU02:FS00:0x15:Project: 7624 (Run 684, Clone 0, Gen 39)
16:15:32:WU02:FS00:0x15:
16:15:32:WU02:FS00:0x15:Assembly optimizations on if available.
16:15:32:WU02:FS00:0x15:Entering M.D.
16:15:34:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  2263893614 573565463 1194305807 1390631238 1763632399
16:15:34:WU02:FS00:0x15:GPU device id=0
16:15:34:WU02:FS00:0x15:Working on Protein
16:15:34:WU02:FS00:0x15:Client config unavailable.
16:15:34:WU02:FS00:0x15:Starting GUI Server
16:16:40:WU02:FS00:0x15:Setting checkpoint frequency: 400000
16:16:40:WARNING:WU02:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
16:16:40:WU02:FS00:0x15:Completed         3 out of 40000000 steps (0%).
16:24:45:WU02:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
16:32:50:WU02:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
16:40:56:WU02:FS00:0x15:Completed   1200000 out of 40000000 steps (3%).
16:47:41:WU02:FS00:0x15:Completed   1600000 out of 40000000 steps (4%).
16:47:42:WU02:FS00:0x15:mdrun_gpu returned 52
16:47:42:WU02:FS00:0x15:NANs detected on GPU
16:47:42:WU02:FS00:0x15:
16:47:42:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
16:47:42:WARNING:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
16:47:43:WU02:FS00:Starting
16:47:43:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
16:47:43:WU02:FS00:Started FahCore on PID 4640
16:47:43:WU02:FS00:Core PID:1052
16:47:43:WU02:FS00:FahCore 0x15 started
16:47:43:WU02:FS00:0x15:
16:47:43:WU02:FS00:0x15:*------------------------------*
16:47:43:WU02:FS00:0x15:Folding@Home GPU Core
16:47:43:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
16:47:43:WU02:FS00:0x15:Build host             AmoebaRemote
16:47:43:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
16:47:43:WU02:FS00:0x15:Core                   15
16:47:43:WU02:FS00:0x15:
16:47:43:WU02:FS00:0x15:Window's signal control handler registered.
16:47:43:WU02:FS00:0x15:Preparing to commence simulation
16:47:43:WU02:FS00:0x15:- Looking at optimizations...
16:47:43:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
16:47:43:WU02:FS00:0x15:- Created dyn
16:47:43:WU02:FS00:0x15:- Files status OK
16:47:43:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
16:47:43:WU02:FS00:0x15:- Expanded 126109 -> 507182 (decompressed 402.1 percent)
16:47:43:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=126109 data_size=507182, decompressed_data_size=507182 diff=0
16:47:43:WU02:FS00:0x15:- Digital signature verified
16:47:43:WU02:FS00:0x15:
16:47:43:WU02:FS00:0x15:Project: 7624 (Run 684, Clone 0, Gen 39)
16:47:43:WU02:FS00:0x15:
16:47:43:WU02:FS00:0x15:Assembly optimizations on if available.
16:47:43:WU02:FS00:0x15:Entering M.D.
16:47:45:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  2263893614 573565463 1194305807 1390631238 1763632399
16:47:45:WU02:FS00:0x15:GPU device id=0
16:47:45:WU02:FS00:0x15:Working on Protein
16:47:45:WU02:FS00:0x15:Client config unavailable.
16:47:45:WU02:FS00:0x15:Starting GUI Server
16:48:51:WU02:FS00:0x15:Setting checkpoint frequency: 400000
16:48:51:WU02:FS00:0x15:Completed         3 out of 40000000 steps (0%).
16:48:52:WARNING:WU02:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
16:56:56:WU02:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
17:04:34:WU02:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
17:04:35:WU02:FS00:0x15:mdrun_gpu returned 52
17:04:35:WU02:FS00:0x15:NANs detected on GPU
17:04:35:WU02:FS00:0x15:
17:04:35:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
17:04:35:WARNING:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
17:04:35:WU02:FS00:Starting
17:04:35:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
17:04:35:WU02:FS00:Started FahCore on PID 2268
17:04:35:WU02:FS00:Core PID:4044
17:04:35:WU02:FS00:FahCore 0x15 started
17:04:36:WU02:FS00:0x15:
17:04:36:WU02:FS00:0x15:*------------------------------*
17:04:36:WU02:FS00:0x15:Folding@Home GPU Core
17:04:36:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
17:04:36:WU02:FS00:0x15:Build host             AmoebaRemote
17:04:36:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
17:04:36:WU02:FS00:0x15:Core                   15
17:04:36:WU02:FS00:0x15:
17:04:36:WU02:FS00:0x15:Window's signal control handler registered.
17:04:36:WU02:FS00:0x15:Preparing to commence simulation
17:04:36:WU02:FS00:0x15:- Looking at optimizations...
17:04:36:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
17:04:36:WU02:FS00:0x15:- Created dyn
17:04:36:WU02:FS00:0x15:- Files status OK
17:04:36:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
17:04:36:WU02:FS00:0x15:- Expanded 126109 -> 507182 (decompressed 402.1 percent)
17:04:36:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=126109 data_size=507182, decompressed_data_size=507182 diff=0
17:04:36:WU02:FS00:0x15:- Digital signature verified
17:04:36:WU02:FS00:0x15:
17:04:36:WU02:FS00:0x15:Project: 7624 (Run 684, Clone 0, Gen 39)
17:04:36:WU02:FS00:0x15:
17:04:36:WU02:FS00:0x15:Assembly optimizations on if available.
17:04:36:WU02:FS00:0x15:Entering M.D.
17:04:38:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  2263893614 573565463 1194305807 1390631238 1763632399
17:04:38:WU02:FS00:0x15:GPU device id=0
17:04:38:WU02:FS00:0x15:Working on Protein
17:04:38:WU02:FS00:0x15:Client config unavailable.
17:04:38:WU02:FS00:0x15:Starting GUI Server
17:05:44:WU02:FS00:0x15:Setting checkpoint frequency: 400000
17:05:44:WARNING:WU02:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
17:05:44:WU02:FS00:0x15:Completed         3 out of 40000000 steps (0%).
17:13:27:WU02:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
17:13:28:WU02:FS00:0x15:mdrun_gpu returned 52
17:13:28:WU02:FS00:0x15:NANs detected on GPU
17:13:28:WU02:FS00:0x15:
17:13:28:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
17:13:28:WARNING:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
17:13:28:WU02:FS00:Starting
17:13:28:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
17:13:28:WU02:FS00:Started FahCore on PID 1088
17:13:28:WU02:FS00:Core PID:3528
17:13:28:WU02:FS00:FahCore 0x15 started
17:13:29:WU02:FS00:0x15:
17:13:29:WU02:FS00:0x15:*------------------------------*
17:13:29:WU02:FS00:0x15:Folding@Home GPU Core
17:13:29:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
17:13:29:WU02:FS00:0x15:Build host             AmoebaRemote
17:13:29:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
17:13:29:WU02:FS00:0x15:Core                   15
17:13:29:WU02:FS00:0x15:
17:13:29:WU02:FS00:0x15:Window's signal control handler registered.
17:13:29:WU02:FS00:0x15:Preparing to commence simulation
17:13:29:WU02:FS00:0x15:- Looking at optimizations...
17:13:29:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
17:13:29:WU02:FS00:0x15:- Created dyn
17:13:29:WU02:FS00:0x15:- Files status OK
17:13:29:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
17:13:29:WU02:FS00:0x15:- Expanded 126109 -> 507182 (decompressed 402.1 percent)
17:13:29:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=126109 data_size=507182, decompressed_data_size=507182 diff=0
17:13:29:WU02:FS00:0x15:- Digital signature verified
17:13:29:WU02:FS00:0x15:
17:13:29:WU02:FS00:0x15:Project: 7624 (Run 684, Clone 0, Gen 39)
17:13:29:WU02:FS00:0x15:
17:13:29:WU02:FS00:0x15:Assembly optimizations on if available.
17:13:29:WU02:FS00:0x15:Entering M.D.
17:13:30:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  2263893614 573565463 1194305807 1390631238 1763632399
17:13:30:WU02:FS00:0x15:GPU device id=0
17:13:30:WU02:FS00:0x15:Working on Protein
17:13:30:WU02:FS00:0x15:Client config unavailable.
17:13:30:WU02:FS00:0x15:Starting GUI Server
17:14:36:WU02:FS00:0x15:Setting checkpoint frequency: 400000
17:14:36:WU02:FS00:0x15:Completed         3 out of 40000000 steps (0%).
17:14:37:WARNING:WU02:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
17:22:41:WU02:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
17:29:34:WU02:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
17:29:34:WU02:FS00:0x15:mdrun_gpu returned 52
17:29:34:WU02:FS00:0x15:NANs detected on GPU
17:29:34:WU02:FS00:0x15:
17:29:34:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
17:29:35:WARNING:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
17:29:35:WARNING:WU02:FS00:Too many errors, failing
17:29:35:WU02:FS00:Sending unit results: id:02 state:SEND error:FAILED project:7624 run:684 clone:0 gen:39 core:0x15 unit:0x00000030664f2dd14fe61478c872b8ae
17:29:35:WU02:FS00:Connecting to 171.64.65.105:8080
17:29:35:WU00:FS00:Connecting to assign-GPU.stanford.edu:80
17:29:35:WU00:FS00:News: Welcome to Folding@Home
17:29:35:WU00:FS00:Assigned to work server 171.64.65.105
17:29:35:WU00:FS00:Requesting new work unit for slot 00: READY gpu:0:GF114 [GeForce GTX 560 Ti] from 171.64.65.105
17:29:35:WU00:FS00:Connecting to 171.64.65.105:8080
17:29:39:WU02:FS00:Server responded WORK_QUIT (404)
17:29:39:WARNING:WU02:FS00:Server did not like results, dumping
17:29:39:WU02:FS00:Cleaning up
17:29:39:WU00:FS00:Downloading 121.90KiB
17:29:39:WU00:FS00:Download complete
17:29:40:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:7625 run:532 clone:3 gen:8 core:0x15 unit:0x0000000b664f2dd14fe61832dcae8e0b
17:29:40:WU00:FS00:Starting
17:29:40:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
17:29:40:WU00:FS00:Started FahCore on PID 4768
17:29:40:WU00:FS00:Core PID:3660
17:29:40:WU00:FS00:FahCore 0x15 started
17:29:40:WU00:FS00:0x15:
17:29:40:WU00:FS00:0x15:*------------------------------*
17:29:40:WU00:FS00:0x15:Folding@Home GPU Core
17:29:40:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
17:29:40:WU00:FS00:0x15:Build host             AmoebaRemote
17:29:40:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
17:29:40:WU00:FS00:0x15:Core                   15
17:29:40:WU00:FS00:0x15:
17:29:40:WU00:FS00:0x15:Window's signal control handler registered.
17:29:40:WU00:FS00:0x15:Preparing to commence simulation
17:29:40:WU00:FS00:0x15:- Looking at optimizations...
17:29:40:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
17:29:40:WU00:FS00:0x15:- Created dyn
17:29:40:WU00:FS00:0x15:- Files status OK
17:29:40:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
17:29:40:WU00:FS00:0x15:- Expanded 124310 -> 502918 (decompressed 404.5 percent)
17:29:40:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=124310 data_size=502918, decompressed_data_size=502918 diff=0
17:29:40:WU00:FS00:0x15:- Digital signature verified
17:29:40:WU00:FS00:0x15:
17:29:40:WU00:FS00:0x15:Project: 7625 (Run 532, Clone 3, Gen 8)
17:29:40:WU00:FS00:0x15:
17:29:40:WU00:FS00:0x15:Assembly optimizations on if available.
17:29:40:WU00:FS00:0x15:Entering M.D.
17:29:42:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  1898919182 1522898141 3223176113 1483437633 2855513497
17:29:42:WU00:FS00:0x15:GPU device id=0
17:29:42:WU00:FS00:0x15:Working on Protein
17:29:42:WU00:FS00:0x15:Client config unavailable.
17:29:42:WU00:FS00:0x15:Starting GUI Server
17:30:48:WU00:FS00:0x15:Setting checkpoint frequency: 400000
17:30:48:WU00:FS00:0x15:Completed         3 out of 40000000 steps (0%).
17:38:53:WU00:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
17:46:52:WU00:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
17:46:53:WU00:FS00:0x15:mdrun_gpu returned 52
17:46:53:WU00:FS00:0x15:NANs detected on GPU
17:46:53:WU00:FS00:0x15:
17:46:53:WU00:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
17:46:53:WARNING:WU00:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
17:46:53:WU00:FS00:Starting
17:46:53:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
17:46:53:WU00:FS00:Started FahCore on PID 1064
17:46:53:WU00:FS00:Core PID:480
17:46:53:WU00:FS00:FahCore 0x15 started
17:46:54:WU00:FS00:0x15:
17:46:54:WU00:FS00:0x15:*------------------------------*
17:46:54:WU00:FS00:0x15:Folding@Home GPU Core
17:46:54:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
17:46:54:WU00:FS00:0x15:Build host             AmoebaRemote
17:46:54:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
17:46:54:WU00:FS00:0x15:Core                   15
17:46:54:WU00:FS00:0x15:
17:46:54:WU00:FS00:0x15:Window's signal control handler registered.
17:46:54:WU00:FS00:0x15:Preparing to commence simulation
17:46:54:WU00:FS00:0x15:- Looking at optimizations...
17:46:54:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
17:46:54:WU00:FS00:0x15:- Created dyn
17:46:54:WU00:FS00:0x15:- Files status OK
17:46:54:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
17:46:54:WU00:FS00:0x15:- Expanded 124310 -> 502918 (decompressed 404.5 percent)
17:46:54:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=124310 data_size=502918, decompressed_data_size=502918 diff=0
17:46:54:WU00:FS00:0x15:- Digital signature verified
17:46:54:WU00:FS00:0x15:
17:46:54:WU00:FS00:0x15:Project: 7625 (Run 532, Clone 3, Gen 8)
17:46:54:WU00:FS00:0x15:
17:46:54:WU00:FS00:0x15:Assembly optimizations on if available.
17:46:54:WU00:FS00:0x15:Entering M.D.
17:46:56:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  1898919182 1522898141 3223176113 1483437633 2855513497
17:46:56:WU00:FS00:0x15:GPU device id=0
17:46:56:WU00:FS00:0x15:Working on Protein
17:46:56:WU00:FS00:0x15:Client config unavailable.
17:46:56:WU00:FS00:0x15:Starting GUI Server
17:48:02:WU00:FS00:0x15:Setting checkpoint frequency: 400000
17:48:02:WARNING:WU00:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
17:48:02:WU00:FS00:0x15:Completed         3 out of 40000000 steps (0%).
17:56:06:WU00:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
******************************** Date: 08/01/13 ********************************
18:04:12:WU00:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
18:12:18:WU00:FS00:0x15:Completed   1200000 out of 40000000 steps (3%).
18:20:23:WU00:FS00:0x15:Completed   1600000 out of 40000000 steps (4%).
18:28:29:WU00:FS00:0x15:Completed   2000000 out of 40000000 steps (5%).
18:35:48:WU00:FS00:0x15:Completed   2400000 out of 40000000 steps (6%).
18:35:48:WU00:FS00:0x15:mdrun_gpu returned 52
18:35:48:WU00:FS00:0x15:NANs detected on GPU
18:35:48:WU00:FS00:0x15:
18:35:48:WU00:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
18:35:49:WARNING:WU00:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
18:35:49:WU00:FS00:Starting
18:35:49:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
18:35:49:WU00:FS00:Started FahCore on PID 3936
18:35:49:WU00:FS00:Core PID:4768
18:35:49:WU00:FS00:FahCore 0x15 started
18:35:50:WU00:FS00:0x15:
18:35:50:WU00:FS00:0x15:*------------------------------*
18:35:50:WU00:FS00:0x15:Folding@Home GPU Core
18:35:50:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
18:35:50:WU00:FS00:0x15:Build host             AmoebaRemote
18:35:50:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
18:35:50:WU00:FS00:0x15:Core                   15
18:35:50:WU00:FS00:0x15:
18:35:50:WU00:FS00:0x15:Window's signal control handler registered.
18:35:50:WU00:FS00:0x15:Preparing to commence simulation
18:35:50:WU00:FS00:0x15:- Looking at optimizations...
18:35:50:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
18:35:50:WU00:FS00:0x15:- Created dyn
18:35:50:WU00:FS00:0x15:- Files status OK
18:35:50:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
18:35:50:WU00:FS00:0x15:- Expanded 124310 -> 502918 (decompressed 404.5 percent)
18:35:50:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=124310 data_size=502918, decompressed_data_size=502918 diff=0
18:35:50:WU00:FS00:0x15:- Digital signature verified
18:35:50:WU00:FS00:0x15:
18:35:50:WU00:FS00:0x15:Project: 7625 (Run 532, Clone 3, Gen 8)
18:35:50:WU00:FS00:0x15:
18:35:50:WU00:FS00:0x15:Assembly optimizations on if available.
18:35:50:WU00:FS00:0x15:Entering M.D.
18:35:51:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  1898919182 1522898141 3223176113 1483437633 2855513497
18:35:51:WU00:FS00:0x15:GPU device id=0
18:35:51:WU00:FS00:0x15:Working on Protein
18:35:51:WU00:FS00:0x15:Client config unavailable.
18:35:51:WU00:FS00:0x15:Starting GUI Server
18:36:57:WU00:FS00:0x15:Setting checkpoint frequency: 400000
18:36:57:WU00:FS00:0x15:Completed         3 out of 40000000 steps (0%).
18:36:58:WARNING:WU00:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
18:45:02:WU00:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
18:51:53:WU00:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
18:51:53:WU00:FS00:0x15:mdrun_gpu returned 52
18:51:53:WU00:FS00:0x15:NANs detected on GPU
18:51:53:WU00:FS00:0x15:
18:51:53:WU00:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
18:51:54:WARNING:WU00:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
18:51:54:WU00:FS00:Starting
18:51:54:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
18:51:54:WU00:FS00:Started FahCore on PID 4488
18:51:54:WU00:FS00:Core PID:1964
18:51:54:WU00:FS00:FahCore 0x15 started
18:51:54:WU00:FS00:0x15:
18:51:54:WU00:FS00:0x15:*------------------------------*
18:51:54:WU00:FS00:0x15:Folding@Home GPU Core
18:51:54:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
18:51:54:WU00:FS00:0x15:Build host             AmoebaRemote
18:51:54:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
18:51:54:WU00:FS00:0x15:Core                   15
18:51:54:WU00:FS00:0x15:
18:51:54:WU00:FS00:0x15:Window's signal control handler registered.
18:51:54:WU00:FS00:0x15:Preparing to commence simulation
18:51:54:WU00:FS00:0x15:- Looking at optimizations...
18:51:54:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
18:51:54:WU00:FS00:0x15:- Created dyn
18:51:54:WU00:FS00:0x15:- Files status OK
18:51:54:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
18:51:54:WU00:FS00:0x15:- Expanded 124310 -> 502918 (decompressed 404.5 percent)
18:51:54:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=124310 data_size=502918, decompressed_data_size=502918 diff=0
18:51:54:WU00:FS00:0x15:- Digital signature verified
18:51:54:WU00:FS00:0x15:
18:51:54:WU00:FS00:0x15:Project: 7625 (Run 532, Clone 3, Gen 8)
18:51:54:WU00:FS00:0x15:
18:51:54:WU00:FS00:0x15:Assembly optimizations on if available.
18:51:54:WU00:FS00:0x15:Entering M.D.
18:51:56:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  1898919182 1522898141 3223176113 1483437633 2855513497
18:51:56:WU00:FS00:0x15:GPU device id=0
18:51:56:WU00:FS00:0x15:Working on Protein
18:51:56:WU00:FS00:0x15:Client config unavailable.
18:51:56:WU00:FS00:0x15:Starting GUI Server
18:53:02:WU00:FS00:0x15:Setting checkpoint frequency: 400000
18:53:02:WU00:FS00:0x15:Completed         3 out of 40000000 steps (0%).
18:53:03:WARNING:WU00:FS00:Detected clock skew (1 mins 09 secs), adjusting time estimates
19:00:05:WU00:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
19:00:06:WU00:FS00:0x15:mdrun_gpu returned 52
19:00:06:WU00:FS00:0x15:NANs detected on GPU
19:00:06:WU00:FS00:0x15:
19:00:06:WU00:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
19:00:06:WARNING:WU00:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
19:00:06:WU00:FS00:Starting
19:00:06:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Admin/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 702 -lifeline 3288 -checkpoint 30 -gpu 0
19:00:06:WU00:FS00:Started FahCore on PID 2620
19:00:06:WU00:FS00:Core PID:4420
19:00:06:WU00:FS00:FahCore 0x15 started
19:00:07:WU00:FS00:0x15:
19:00:07:WU00:FS00:0x15:*------------------------------*
19:00:07:WU00:FS00:0x15:Folding@Home GPU Core
19:00:07:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
19:00:07:WU00:FS00:0x15:Build host             AmoebaRemote
19:00:07:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
19:00:07:WU00:FS00:0x15:Core                   15
19:00:07:WU00:FS00:0x15:
19:00:07:WU00:FS00:0x15:Window's signal control handler registered.
19:00:07:WU00:FS00:0x15:Preparing to commence simulation
19:00:07:WU00:FS00:0x15:- Looking at optimizations...
19:00:07:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
19:00:07:WU00:FS00:0x15:- Created dyn
19:00:07:WU00:FS00:0x15:- Files status OK
19:00:07:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
19:00:07:WU00:FS00:0x15:- Expanded 124310 -> 502918 (decompressed 404.5 percent)
19:00:07:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=124310 data_size=502918, decompressed_data_size=502918 diff=0
19:00:07:WU00:FS00:0x15:- Digital signature verified
19:00:07:WU00:FS00:0x15:
19:00:07:WU00:FS00:0x15:Project: 7625 (Run 532, Clone 3, Gen 8)
19:00:07:WU00:FS00:0x15:
19:00:07:WU00:FS00:0x15:Assembly optimizations on if available.
19:00:07:WU00:FS00:0x15:Entering M.D.
19:00:09:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  1898919182 1522898141 3223176113 1483437633 2855513497
19:00:09:WU00:FS00:0x15:GPU device id=0
19:00:09:WU00:FS00:0x15:Working on Protein
19:00:09:WU00:FS00:0x15:Client config unavailable.
19:00:09:WU00:FS00:0x15:Starting GUI Server
19:01:15:WARNING:WU00:FS00:Detected clock skew (1 mins 08 secs), adjusting time estimates
19:01:15:WU00:FS00:0x15:Setting checkpoint frequency: 400000
19:01:15:WU00:FS00:0x15:Completed         3 out of 40000000 steps (0%).
19:09:20:WU00:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
19:17:25:WU00:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
19:24:26:WU00:FS00:0x15:Completed   1200000 out of 40000000 steps (3%).
19:24:27:WU00:FS00:0x15:mdrun_gpu returned 52
19:24:27:WU00:FS00:0x15:NANs detected on GPU
19:24:27:WU00:FS00:0x15:
19:24:27:WU00:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
19:24:27:WARNING:WU00:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
19:24:27:WARNING:WU00:FS00:Too many errors, failing
19:24:27:WU00:FS00:Sending unit results: id:00 state:SEND error:FAILED project:7625 run:532 clone:3 gen:8 core:0x15 unit:0x0000000b664f2dd14fe61832dcae8e0b
19:24:27:WU00:FS00:Connecting to 171.64.65.105:8080
19:24:28:WU00:FS00:Server responded WORK_QUIT (404)
19:24:28:WARNING:WU00:FS00:Server did not like results, dumping
19:24:28:WU00:FS00:Cleaning up

A bunch of UNSTABLE_MACHINES. My slot failed again. I'm pretty sure I installed my card correctly so I'll temporarily remove the beta switch and see if I continue to get errors.

[art_l_j_PlanetAMD64]

A bunch of UNSTABLE_MACHINES. My slot failed again. I'm pretty sure I installed my card correctly so I'll temporarily remove the beta switch and see if I continue to get errors.

I don't think removing the beta switch will make any difference, these WUs have been in general release for some time.

I did a grep for 'FahCore returned:' on all the log files for my dual-GTX460 system that has processed a lot of the 762x WUs, and every result is like this:
10:52:29:WU03:FS02:FahCore returned: FINISHED_UNIT (100 = 0x64)

No errors of any kind.

[k1wi]

Are you getting errors on any other GPU projects? (i.e. are you just getting these projects or are you getting non 762x projects?)

Sounds like the problem is with the GPU, either because it's not up to scratch at all or because these projects pusher it harder than it can cope (even at stock speeds). But I'm sure a mod could confirm whether anyone else has returned the projects successfully.

This thread is in a open forum by the way guys, so we shouldn't be mentioning switches or flags.

[art_l_j_PlanetAMD64]

My rig with the dual GTX460s finished the 7623s and picked up a couple of 7626s, so the 762x WUs are not completely gone, yet. Hooray!

[Jesse_V]

Are you getting errors on any other GPU projects? (i.e. are you just getting these projects or are you getting non 762x projects?)

I've been getting mostly these projects ever since I reinstalled the card on Monday. I'm getting other projects now and so far they've been going smoothly.

Sounds like the problem is with the GPU, either because it's not up to scratch at all or because these projects pusher it harder than it can cope (even at stock speeds). But I'm sure a mod could confirm whether anyone else has returned the projects successfully.

Since it's only been a few days since I reinstalled the card after getting it back from an RMA, the variables of problems with the project and problems with the card are kind of mixed right now. That's why I switched to standard WUs to try to isolate the culprit.

This thread is in a open forum by the way guys, so we shouldn't be mentioning switches or flags.

Good catch. My apologies.

[art_l_j_PlanetAMD64]

My rig with the dual GTX460s finished the two 7626s and picked up a 7623 and a 7625, so maybe the lack of available 762x WUs was only temporary? I sure hope so!

[art_l_j_PlanetAMD64]

Well, it seems like the lack of 762x WUs was only temporary. The 7623 and 7625 WUs just completed, and the dual-GTX460 rig now has a 7625 and a 7626. Maybe a server was down for maintenance, or maybe another batch of 762x WUs were just being set up and were not available at the time the P8073s were being picked up. In any case, I'm glad to see all those 762x WUs coming my way, Thanks PG!

[kscott]

Why was this core forced on us?

[P5-133XL]

Because they want Kepler's to be able to fold and not have to maintain separate cores for Kepler's and non-Kepler video cards. That would be my speculation as to motives.

[codysluder]

Why was this core forced on us?

Because it can do things that earlier versions can't. (That would be my speculation.)

[kscott]

Why was this core forced on us?

Because it can do things that earlier versions can't. (That would be my speculation.)

Like what other then not being optimized for Fermi and letting Kepler fold