Odd fail pattern 10505 (Run 1xx, Clone 33, Gen 5) +another

Moderators: Site Moderators, FAHC Science Team

Post Reply
rhavern
Posts: 425
Joined: Mon Dec 03, 2007 8:45 am
Location: UK

Odd fail pattern 10505 (Run 1xx, Clone 33, Gen 5) +another

Post by rhavern »

Long term dedicated folding box, XP-32, GTS250 in an open case with loads of cooling in a hosting center. Odd pattern of failures in same Clone/Gen, I don't know enough to say if this is normally how the servers build successive RCGs.

[05:51:44] Project: 10505 (Run 109, Clone 33, Gen 5)
[05:52:05] Project: 10505 (Run 111, Clone 33, Gen 5)
[05:52:31] Project: 10505 (Run 112, Clone 33, Gen 5)
[05:52:56] Project: 10505 (Run 113, Clone 33, Gen 5)
[05:53:21] Project: 10505 (Run 114, Clone 33, Gen 5)

I could no longer access the box via logmein, had to reboot to regain control, so that may be a factor as well.

Here is the log excerpt:

Code: Select all

# Windows GPU Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.41r2

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\fah-gpu1
Service: C:\fah-gpu1\Folding@home-Win32-GPU.exe
Arguments: -svcstart -d C:\fah-gpu1 -verbosity 9 -gpu 0  

Launched as a service.
Entered C:\fah-gpu1 to do work.

[05:51:42] Connecting to http://assign-GPU.stanford.edu:8080/
[05:51:43] Posted data.
[05:51:43] Initial: 43AB; - Successful: assigned to (171.67.108.21).
[05:51:43] + News From Folding@Home: Welcome to Folding@Home
[05:51:43] Loaded queue successfully.
[05:51:43] Gpu type=2 species=11.
[05:51:43] Sent data
[05:51:43] Connecting to http://171.67.108.21:8080/
[05:51:44] Posted data.
[05:51:44] Initial: 0000; - Receiving payload (expected size: 19800)
[05:51:44] Conversation time very short, giving reduced weight in bandwidth avg
[05:51:44] - Downloaded at ~38 kB/s
[05:51:44] - Averaged speed for that direction ~40 kB/s
[05:51:44] + Received work.
[05:51:44] Trying to send all finished work units
[05:51:44] + No unsent completed units remaining.
[05:51:44] + Closed connections
[05:51:44] 
[05:51:44] + Processing work unit
[05:51:44] Core required: FahCore_11.exe
[05:51:44] Core found.
[05:51:44] Working on queue slot 08 [June 13 05:51:44 UTC]
[05:51:44] + Working ...
[05:51:44] - Calling '.\FahCore_11.exe -dir work/ -suffix 08 -nice 19 -priority 96 -nocpulock -checkpoint 15 

-service -verbose -lifeline 900 -version 641'

[05:51:44] 
[05:51:44] *------------------------------*
[05:51:44] Folding@Home GPU Core
[05:51:44] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[05:51:44] 
[05:51:44] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[05:51:44] Build host: amoeba
[05:51:44] Board Type: Nvidia
[05:51:44] Core      : 
[05:51:44] Preparing to commence simulation
[05:51:44] - Looking at optimizations...
[05:51:44] DeleteFrameFiles: successfully deleted file=work/wudata_08.ckp
[05:51:44] - Created dyn
[05:51:44] - Files status OK
[05:51:44] - Expanded 19288 -> 133151 (decompressed 690.3 percent)
[05:51:44] Called DecompressByteArray: compressed_data_size=19288 data_size=133151, decompressed_data_size=133151 

diff=0
[05:51:44] - Digital signature verified
[05:51:44] 
[05:51:44] Project: 10505 (Run 109, Clone 33, Gen 5)
[05:51:44] 
[05:51:44] Assembly optimizations on if available.
[05:51:44] Entering M.D.
[05:51:50] Tpr hash work/wudata_08.tpr:  3764075888 2056488036 1692200233 1752404722 2690882287
[05:51:50] 
[05:51:50] Calling fah_main args: 14 usage=100
[05:51:50] 
[05:51:50] Run: exception thrown during GuardedRun
[05:51:50] Run: exception thrown in GuardedRun -- Gromacs cannot continue further.
[05:51:50] Going to send back what have done -- stepsTotalG=0
[05:51:50] Work fraction=0.0000 steps=0.
[05:51:54] logfile size=4943 infoLength=4943 edr=0 trr=23
[05:51:54] + Opened results file
[05:51:54] - Writing 5479 bytes of core data to disk...
[05:51:54] Decompress: Memory problem 2
[05:51:54] - Compressed data size (2089877597) exceeds limit. 
[05:51:54] DeleteFrameFiles: successfully deleted file=work/wudata_08.ckp
[05:51:55] 
[05:51:55] Folding@home Core Shutdown: UNSTABLE_MACHINE
[05:51:58] CoreStatus = 7A (122)
[05:51:58] Sending work to server
[05:51:58] Project: 10505 (Run 109, Clone 33, Gen 5)
[05:51:58] - Read packet limit of 540015616... Set to 524286976.


[05:51:58] + Attempting to send results [June 13 05:51:58 UTC]
[05:51:58] - Reading file work/wuresults_08.dat from core
[05:51:58]   (Read 512 bytes from disk)
[05:51:58] Gpu type=2 species=11.
[05:51:58] Connecting to http://171.67.108.21:8080/
[05:51:59] Posted data.
[05:51:59] Initial: 0000; - Uploaded at ~1 kB/s
[05:51:59] - Averaged speed for that direction ~45 kB/s
[05:51:59] - Server reports problem with unit.
[05:51:59] Trying to send all finished work units
[05:51:59] + No unsent completed units remaining.
[05:51:59] - Preparing to get new work unit...
[05:51:59] Cleaning up work directory
[05:51:59] + Attempting to get work packet
[05:51:59] Passkey found
[05:51:59] - Will indicate memory of 2046 MB
[05:51:59] Gpu type=2 species=11.
[05:51:59] - Connecting to assignment server
[05:51:59] Connecting to http://assign-GPU.stanford.edu:8080/
[05:51:59] Posted data.
[05:51:59] Initial: 43AB; - Successful: assigned to (171.67.108.21).
[05:51:59] + News From Folding@Home: Welcome to Folding@Home
[05:51:59] Loaded queue successfully.
[05:51:59] Gpu type=2 species=11.
[05:51:59] Sent data
[05:51:59] Connecting to http://171.67.108.21:8080/
[05:52:00] Posted data.
[05:52:00] Initial: 0000; - Receiving payload (expected size: 19434)
[05:52:00] Conversation time very short, giving reduced weight in bandwidth avg
[05:52:00] - Downloaded at ~37 kB/s
[05:52:00] - Averaged speed for that direction ~40 kB/s
[05:52:00] + Received work.
[05:52:00] Trying to send all finished work units
[05:52:00] + No unsent completed units remaining.
[05:52:00] + Closed connections
[05:52:05] 
[05:52:05] + Processing work unit
[05:52:05] Core required: FahCore_11.exe
[05:52:05] Core found.
[05:52:05] Working on queue slot 09 [June 13 05:52:05 UTC]
[05:52:05] + Working ...
[05:52:05] - Calling '.\FahCore_11.exe -dir work/ -suffix 09 -nice 19 -priority 96 -nocpulock -checkpoint 15 

-service -verbose -lifeline 900 -version 641'

[05:52:05] 
[05:52:05] *------------------------------*
[05:52:05] Folding@Home GPU Core
[05:52:05] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[05:52:05] 
[05:52:05] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[05:52:05] Build host: amoeba
[05:52:05] Board Type: Nvidia
[05:52:05] Core      : 
[05:52:05] Preparing to commence simulation
[05:52:05] - Looking at optimizations...
[05:52:05] DeleteFrameFiles: successfully deleted file=work/wudata_09.ckp
[05:52:05] - Created dyn
[05:52:05] - Files status OK
[05:52:05] - Expanded 18922 -> 131011 (decompressed 692.3 percent)
[05:52:05] Called DecompressByteArray: compressed_data_size=18922 data_size=131011, decompressed_data_size=131011 

diff=0
[05:52:05] - Digital signature verified
[05:52:05] 
[05:52:05] Project: 10505 (Run 111, Clone 33, Gen 5)
[05:52:05] 
[05:52:06] Assembly optimizations on if available.
[05:52:06] Entering M.D.
[05:52:12] Tpr hash work/wudata_09.tpr:  1614437898 436440590 3907527310 227648155 2761889323
[05:52:12] 
[05:52:12] Calling fah_main args: 14 usage=100
[05:52:12] 
[05:52:12] Run: exception thrown during GuardedRun
[05:52:12] Run: exception thrown in GuardedRun -- Gromacs cannot continue further.
[05:52:12] Going to send back what have done -- stepsTotalG=0
[05:52:12] Work fraction=0.0000 steps=0.
[05:52:16] logfile size=4943 infoLength=4943 edr=0 trr=23
[05:52:16] + Opened results file
[05:52:16] - Writing 5479 bytes of core data to disk...
[05:52:16] Done: 4967 -> 1845 (compressed to 37.1 percent)
[05:52:16]   ... Done.
[05:52:16] DeleteFrameFiles: successfully deleted file=work/wudata_09.ckp
[05:52:16] 
[05:52:16] Folding@home Core Shutdown: UNSTABLE_MACHINE
[05:52:19] CoreStatus = 7A (122)
[05:52:19] Sending work to server
[05:52:19] Project: 10505 (Run 111, Clone 33, Gen 5)
[05:52:19] - Read packet limit of 540015616... Set to 524286976.


[05:52:19] + Attempting to send results [June 13 05:52:19 UTC]
[05:52:19] - Reading file work/wuresults_09.dat from core
[05:52:19]   (Read 2357 bytes from disk)
[05:52:19] Gpu type=2 species=11.
[05:52:19] Connecting to http://171.67.108.21:8080/
[05:52:20] Posted data.
[05:52:20] Initial: 0000; - Uploaded at ~3 kB/s
[05:52:20] - Averaged speed for that direction ~36 kB/s
[05:52:20] + Results successfully sent
[05:52:20] Thank you for your contribution to Folding@Home.
[05:52:24] Trying to send all finished work units
[05:52:24] + No unsent completed units remaining.
[05:52:24] - Preparing to get new work unit...
[05:52:24] Cleaning up work directory
[05:52:24] + Attempting to get work packet
[05:52:24] Passkey found
[05:52:24] - Will indicate memory of 2046 MB
[05:52:24] Gpu type=2 species=11.
[05:52:24] - Connecting to assignment server
[05:52:24] Connecting to http://assign-GPU.stanford.edu:8080/
[05:52:25] Posted data.
[05:52:25] Initial: 43AB; - Successful: assigned to (171.67.108.21).
[05:52:25] + News From Folding@Home: Welcome to Folding@Home
[05:52:25] Loaded queue successfully.
[05:52:25] Gpu type=2 species=11.
[05:52:25] Sent data
[05:52:25] Connecting to http://171.67.108.21:8080/
[05:52:25] Posted data.
[05:52:25] Initial: 0000; - Receiving payload (expected size: 19796)
[05:52:25] Conversation time very short, giving reduced weight in bandwidth avg
[05:52:25] - Downloaded at ~38 kB/s
[05:52:25] - Averaged speed for that direction ~39 kB/s
[05:52:25] + Received work.
[05:52:25] Trying to send all finished work units
[05:52:25] + No unsent completed units remaining.
[05:52:25] + Closed connections
[05:52:30] 
[05:52:30] + Processing work unit
[05:52:30] Core required: FahCore_11.exe
[05:52:30] Core found.
[05:52:30] Working on queue slot 00 [June 13 05:52:30 UTC]
[05:52:30] + Working ...
[05:52:30] - Calling '.\FahCore_11.exe -dir work/ -suffix 00 -nice 19 -priority 96 -nocpulock -checkpoint 15 

-service -verbose -lifeline 900 -version 641'

[05:52:31] 
[05:52:31] *------------------------------*
[05:52:31] Folding@Home GPU Core
[05:52:31] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[05:52:31] 
[05:52:31] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[05:52:31] Build host: amoeba
[05:52:31] Board Type: Nvidia
[05:52:31] Core      : 
[05:52:31] Preparing to commence simulation
[05:52:31] - Looking at optimizations...
[05:52:31] DeleteFrameFiles: successfully deleted file=work/wudata_00.ckp
[05:52:31] - Created dyn
[05:52:31] - Files status OK
[05:52:31] - Expanded 19284 -> 132855 (decompressed 688.9 percent)
[05:52:31] Called DecompressByteArray: compressed_data_size=19284 data_size=132855, decompressed_data_size=132855 

diff=0
[05:52:31] - Digital signature verified
[05:52:31] 
[05:52:31] Project: 10505 (Run 112, Clone 33, Gen 5)
[05:52:31] 
[05:52:31] Assembly optimizations on if available.
[05:52:31] Entering M.D.
[05:52:37] Tpr hash work/wudata_00.tpr:  2920810496 4036697808 4124880660 4075535490 1012700984
[05:52:37] 
[05:52:37] Calling fah_main args: 14 usage=100
[05:52:37] 
[05:52:37] Run: exception thrown during GuardedRun
[05:52:37] Run: exception thrown in GuardedRun -- Gromacs cannot continue further.
[05:52:37] Going to send back what have done -- stepsTotalG=0
[05:52:37] Work fraction=0.0000 steps=0.
[05:52:41] logfile size=4943 infoLength=4943 edr=0 trr=23
[05:52:41] + Opened results file
[05:52:41] - Writing 5479 bytes of core data to disk...
[05:52:41] Done: 4967 -> 1842 (compressed to 37.0 percent)
[05:52:41]   ... Done.
[05:52:41] DeleteFrameFiles: successfully deleted file=work/wudata_00.ckp
[05:52:41] 
[05:52:41] Folding@home Core Shutdown: UNSTABLE_MACHINE
[05:52:45] CoreStatus = 7A (122)
[05:52:45] Sending work to server
[05:52:45] Project: 10505 (Run 112, Clone 33, Gen 5)
[05:52:45] - Read packet limit of 540015616... Set to 524286976.


[05:52:45] + Attempting to send results [June 13 05:52:45 UTC]
[05:52:45] - Reading file work/wuresults_00.dat from core
[05:52:45]   (Read 2354 bytes from disk)
[05:52:45] Gpu type=2 species=11.
[05:52:45] Connecting to http://171.67.108.21:8080/
[05:52:45] Posted data.
[05:52:45] Initial: 0000; Conversation time very short, giving reduced weight in bandwidth avg
[05:52:45] - Uploaded at ~6 kB/s
[05:52:45] - Averaged speed for that direction ~33 kB/s
[05:52:45] + Results successfully sent
[05:52:45] Thank you for your contribution to Folding@Home.
[05:52:49] Trying to send all finished work units
[05:52:49] + No unsent completed units remaining.
[05:52:49] - Preparing to get new work unit...
[05:52:49] Cleaning up work directory
[05:52:49] + Attempting to get work packet
[05:52:49] Passkey found
[05:52:49] - Will indicate memory of 2046 MB
[05:52:49] Gpu type=2 species=11.
[05:52:49] - Connecting to assignment server
[05:52:49] Connecting to http://assign-GPU.stanford.edu:8080/
[05:52:50] Posted data.
[05:52:50] Initial: 43AB; - Successful: assigned to (171.67.108.21).
[05:52:50] + News From Folding@Home: Welcome to Folding@Home
[05:52:50] Loaded queue successfully.
[05:52:50] Gpu type=2 species=11.
[05:52:50] Sent data
[05:52:50] Connecting to http://171.67.108.21:8080/
[05:52:51] Posted data.
[05:52:51] Initial: 0000; - Receiving payload (expected size: 20479)
[05:52:51] Conversation time very short, giving reduced weight in bandwidth avg
[05:52:51] - Downloaded at ~39 kB/s
[05:52:51] - Averaged speed for that direction ~39 kB/s
[05:52:51] + Received work.
[05:52:51] Trying to send all finished work units
[05:52:51] + No unsent completed units remaining.
[05:52:51] + Closed connections
[05:52:56] 
[05:52:56] + Processing work unit
[05:52:56] Core required: FahCore_11.exe
[05:52:56] Core found.
[05:52:56] Working on queue slot 01 [June 13 05:52:56 UTC]
[05:52:56] + Working ...
[05:52:56] - Calling '.\FahCore_11.exe -dir work/ -suffix 01 -nice 19 -priority 96 -nocpulock -checkpoint 15 

-service -verbose -lifeline 900 -version 641'

[05:52:56] 
[05:52:56] *------------------------------*
[05:52:56] Folding@Home GPU Core
[05:52:56] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[05:52:56] 
[05:52:56] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[05:52:56] Build host: amoeba
[05:52:56] Board Type: Nvidia
[05:52:56] Core      : 
[05:52:56] Preparing to commence simulation
[05:52:56] - Looking at optimizations...
[05:52:56] DeleteFrameFiles: successfully deleted file=work/wudata_01.ckp
[05:52:56] - Created dyn
[05:52:56] - Files status OK
[05:52:56] - Expanded 19967 -> 136803 (decompressed 685.1 percent)
[05:52:56] Called DecompressByteArray: compressed_data_size=19967 data_size=136803, decompressed_data_size=136803 

diff=0
[05:52:56] - Digital signature verified
[05:52:56] 
[05:52:56] Project: 10505 (Run 113, Clone 33, Gen 5)
[05:52:56] 
[05:52:56] Assembly optimizations on if available.
[05:52:56] Entering M.D.
[05:53:02] Tpr hash work/wudata_01.tpr:  1109950791 434628533 4235260836 995367434 1155534867
[05:53:02] 
[05:53:02] Calling fah_main args: 14 usage=100
[05:53:02] 
[05:53:02] mdrun_gpu returned 
[05:53:02] Going to send back what have done -- stepsTotalG=0
[05:53:02] Work fraction=0.0000 steps=0.
[05:53:06] logfile size=4943 infoLength=4943 edr=0 trr=25
[05:53:06] + Opened results file
[05:53:06] - Writing 5481 bytes of core data to disk...
[05:53:06] Done: 4969 -> 1837 (compressed to 36.9 percent)
[05:53:06]   ... Done.
[05:53:06] DeleteFrameFiles: successfully deleted file=work/wudata_01.ckp
[05:53:06] 
[05:53:06] Folding@home Core Shutdown: UNSTABLE_MACHINE
[05:53:10] CoreStatus = 7A (122)
[05:53:10] Sending work to server
[05:53:10] Project: 10505 (Run 113, Clone 33, Gen 5)
[05:53:10] - Read packet limit of 540015616... Set to 524286976.


[05:53:10] + Attempting to send results [June 13 05:53:10 UTC]
[05:53:10] - Reading file work/wuresults_01.dat from core
[05:53:10]   (Read 2349 bytes from disk)
[05:53:10] Gpu type=2 species=11.
[05:53:10] Connecting to http://171.67.108.21:8080/
[05:53:10] Posted data.
[05:53:10] Initial: 0000; Conversation time very short, giving reduced weight in bandwidth avg
[05:53:10] - Uploaded at ~6 kB/s
[05:53:10] - Averaged speed for that direction ~30 kB/s
[05:53:10] + Results successfully sent
[05:53:10] Thank you for your contribution to Folding@Home.
[05:53:14] Trying to send all finished work units
[05:53:14] + No unsent completed units remaining.
[05:53:14] - Preparing to get new work unit...
[05:53:14] Cleaning up work directory
[05:53:14] + Attempting to get work packet
[05:53:14] Passkey found
[05:53:14] - Will indicate memory of 2046 MB
[05:53:14] Gpu type=2 species=11.
[05:53:14] - Connecting to assignment server
[05:53:14] Connecting to http://assign-GPU.stanford.edu:8080/
[05:53:15] Posted data.
[05:53:15] Initial: 43AB; - Successful: assigned to (171.67.108.21).
[05:53:15] + News From Folding@Home: Welcome to Folding@Home
[05:53:15] Loaded queue successfully.
[05:53:15] Gpu type=2 species=11.
[05:53:15] Sent data
[05:53:15] Connecting to http://171.67.108.21:8080/
[05:53:16] Posted data.
[05:53:16] Initial: 0000; - Receiving payload (expected size: 20724)
[05:53:16] Conversation time very short, giving reduced weight in bandwidth avg
[05:53:16] - Downloaded at ~40 kB/s
[05:53:16] - Averaged speed for that direction ~39 kB/s
[05:53:16] + Received work.
[05:53:16] Trying to send all finished work units
[05:53:16] + No unsent completed units remaining.
[05:53:16] + Closed connections
[05:53:21] 
[05:53:21] + Processing work unit
[05:53:21] Core required: FahCore_11.exe
[05:53:21] Core found.
[05:53:21] Working on queue slot 02 [June 13 05:53:21 UTC]
[05:53:21] + Working ...
[05:53:21] - Calling '.\FahCore_11.exe -dir work/ -suffix 02 -nice 19 -priority 96 -nocpulock -checkpoint 15 

-service -verbose -lifeline 900 -version 641'

[05:53:21] 
[05:53:21] *------------------------------*
[05:53:21] Folding@Home GPU Core
[05:53:21] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[05:53:21] 
[05:53:21] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[05:53:21] Build host: amoeba
[05:53:21] Board Type: Nvidia
[05:53:21] Core      : 
[05:53:21] Preparing to commence simulation
[05:53:21] - Looking at optimizations...
[05:53:21] DeleteFrameFiles: successfully deleted file=work/wudata_02.ckp
[05:53:21] - Created dyn
[05:53:21] - Files status OK
[05:53:21] - Expanded 20212 -> 137811 (decompressed 681.8 percent)
[05:53:21] Called DecompressByteArray: compressed_data_size=20212 data_size=137811, decompressed_data_size=137811 

diff=0
[05:53:21] - Digital signature verified
[05:53:21] 
[05:53:21] Project: 10505 (Run 114, Clone 33, Gen 5)
[05:53:21] 
[05:53:21] Assembly optimizations on if available.
[05:53:21] Entering M.D.
[05:53:27] Tpr hash work/wudata_02.tpr:  1329702355 2048099502 4161122358 1357212803 1134250591
[05:53:27] 
[05:53:27] Calling fah_main args: 14 usage=100
[05:53:27] 
[05:53:28] Run: exception thrown during GuardedRun
[05:53:28] Run: exception thrown in GuardedRun -- Gromacs cannot continue further.
[05:53:28] Going to send back what have done -- stepsTotalG=0
[05:53:28] Work fraction=0.0000 steps=0.
[05:53:32] logfile size=4943 infoLength=4943 edr=0 trr=23
[05:53:32] + Opened results file
[05:53:32] - Writing 5479 bytes of core data to disk...
[05:53:32] Done: 4967 -> 1846 (compressed to 37.1 percent)
[05:53:32]   ... Done.
[05:53:32] DeleteFrameFiles: successfully deleted file=work/wudata_02.ckp
[05:53:32] 
[05:53:32] Folding@home Core Shutdown: UNSTABLE_MACHINE
[05:53:35] CoreStatus = 7A (122)
[05:53:35] Sending work to server
[05:53:35] Project: 10505 (Run 114, Clone 33, Gen 5)
[05:53:35] - Read packet limit of 540015616... Set to 524286976.


[05:53:35] + Attempting to send results [June 13 05:53:35 UTC]
[05:53:35] - Reading file work/wuresults_02.dat from core
[05:53:35]   (Read 2358 bytes from disk)
[05:53:35] Gpu type=2 species=11.
[05:53:35] Connecting to http://171.67.108.21:8080/
[05:53:36] Posted data.
[05:53:36] Initial: 0000; - Uploaded at ~3 kB/s
[05:53:36] - Averaged speed for that direction ~25 kB/s
[05:53:36] + Results successfully sent
[05:53:36] Thank you for your contribution to Folding@Home.
[05:53:40] EUE limit exceeded. Pausing 24 hours.
[08:28:26] - Autosending finished units... [June 13 08:28:26 UTC]
[08:28:26] Trying to send all finished work units
[08:28:26] + No unsent completed units remaining.
[08:28:26] - Autosend completed
A second GPU (9800GX2) in the same box failed in a similar fashion:

[20:50:43] Project: 10505 (Run 47, Clone 32, Gen 5)
[20:51:12] Project: 10505 (Run 77, Clone 32, Gen 5)
[20:51:33] Project: 10505 (Run 78, Clone 32, Gen 5)
[20:51:55] Project: 10505 (Run 79, Clone 32, Gen 5)
[20:52:20] Project: 10505 (Run 80, Clone 32, Gen 5)
[20:52:41] Project: 10505 (Run 81, Clone 32, Gen 5)
[20:53:02] Project: 10505 (Run 82, Clone 32, Gen 5)
[20:53:23] Project: 10505 (Run 83, Clone 32, Gen 5)

Code: Select all

# Windows GPU Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.41r2

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\fah-gpu3
Service: C:\fah-gpu3\Folding@home-Win32-GPU.exe
Arguments: -svcstart -d C:\fah-gpu3 -verbosity 9 -gpu 2 

Launched as a service.
Entered C:\fah-gpu3 to do work.


[20:50:43] Project: 10505 (Run 47, Clone 32, Gen 5)
[20:50:43] - Read packet limit of 540015616... Set to 524286976.


[20:50:49] Connecting to http://171.67.108.21:8080/
[20:50:50] Posted data.
[20:50:50] Initial: 0000; - Receiving payload (expected size: 23721)
[20:50:50] Conversation time very short, giving reduced weight in bandwidth avg
[20:50:50] - Downloaded at ~46 kB/s
[20:50:50] - Averaged speed for that direction ~41 kB/s
[20:50:50] + Received work.
[20:50:50] Trying to send all finished work units
[20:50:50] + No unsent completed units remaining.
[20:50:50] + Closed connections
[20:50:50] 
[20:50:50] + Processing work unit
[20:50:50] Core required: FahCore_11.exe
[20:50:50] Core found.
[20:50:50] Working on queue slot 08 [June 12 20:50:50 UTC]
[20:50:50] + Working ...
[20:50:50] - Calling '.\FahCore_11.exe -dir work/ -suffix 08 -nice 19 -priority 96 -nocpulock -checkpoint 15 

-service -verbose -lifeline 1608 -version 641'

[20:50:50] 
[20:50:50] *------------------------------*
[20:50:50] Folding@Home GPU Core
[20:50:50] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[20:50:50] 
[20:50:50] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[20:50:50] Build host: amoeba
[20:50:50] Board Type: Nvidia
[20:50:50] Core      : 
[20:50:50] Preparing to commence simulation
[20:50:50] - Looking at optimizations...
[20:50:50] DeleteFrameFiles: successfully deleted file=work/wudata_08.ckp
[20:50:50] - Created dyn
[20:50:50] - Files status OK
[20:50:50] - Expanded 23209 -> 152107 (decompressed 655.3 percent)
[20:50:50] Called DecompressByteArray: compressed_data_size=23209 data_size=152107, decompressed_data_size=152107 

diff=0
[20:50:50] - Digital signature verified
[20:50:50] 
[20:50:50] Project: 10505 (Run 76, Clone 32, Gen 5)
[20:50:50] 
[20:50:50] Assembly optimizations on if available.
[20:50:50] Entering M.D.
[20:50:56] Tpr hash work/wudata_08.tpr:  2129523753 812701376 3482688812 1784594493 3493562770
[20:50:56] 
[20:50:56] Calling fah_main args: 14 usage=100
[20:50:57] 
[20:50:57] Working on Protein
[20:50:57] Client config found, loading data.
[20:50:57] mdrun_gpu returned 
[20:50:57] Going to send back what have done -- stepsTotalG=30000000
[20:50:57] Work fraction=0.0000 steps=30000000.
[20:50:57] Starting GUI Server
[20:51:01] logfile size=9592 infoLength=9592 edr=0 trr=25
[20:51:01] + Opened results file
[20:51:01] - Writing 10130 bytes of core data to disk...
[20:51:01] Decompress: Memory problem 2
[20:51:01] - Compressed data size (2089877597) exceeds limit. 
[20:51:01] DeleteFrameFiles: successfully deleted file=work/wudata_08.ckp
[20:51:01] 
[20:51:01] Folding@home Core Shutdown: EARLY_UNIT_END
[20:51:04] CoreStatus = 72 (114)
[20:51:04] Sending work to server
[20:51:04] Project: 10505 (Run 76, Clone 32, Gen 5)
[20:51:04] - Read packet limit of 540015616... Set to 524286976.


[20:51:04] + Attempting to send results [June 12 20:51:04 UTC]
[20:51:04] - Reading file work/wuresults_08.dat from core
[20:51:04]   (Read 512 bytes from disk)
[20:51:04] Gpu type=2 species=11.
[20:51:04] Connecting to http://171.67.108.21:8080/
[20:51:05] Posted data.
[20:51:05] Initial: 0000; - Uploaded at ~1 kB/s
[20:51:05] - Averaged speed for that direction ~41 kB/s
[20:51:05] - Server reports problem with unit.
[20:51:05] Trying to send all finished work units
[20:51:05] + No unsent completed units remaining.
[20:51:05] - Preparing to get new work unit...
[20:51:05] Cleaning up work directory
[20:51:05] + Attempting to get work packet
[20:51:05] Passkey found
[20:51:05] - Will indicate memory of 2046 MB
[20:51:05] Gpu type=2 species=11.
[20:51:05] - Connecting to assignment server
[20:51:05] Connecting to http://assign-GPU.stanford.edu:8080/
[20:51:06] Posted data.
[20:51:06] Initial: 43AB; - Successful: assigned to (171.67.108.21).
[20:51:06] + News From Folding@Home: Welcome to Folding@Home
[20:51:06] Loaded queue successfully.
[20:51:06] Gpu type=2 species=11.
[20:51:06] Sent data
[20:51:06] Connecting to http://171.67.108.21:8080/
[20:51:06] Posted data.
[20:51:06] Initial: 0000; - Receiving payload (expected size: 23499)
[20:51:06] Conversation time very short, giving reduced weight in bandwidth avg
[20:51:06] - Downloaded at ~45 kB/s
[20:51:06] - Averaged speed for that direction ~41 kB/s
[20:51:06] + Received work.
[20:51:06] Trying to send all finished work units
[20:51:06] + No unsent completed units remaining.
[20:51:06] + Closed connections
[20:51:11] 
[20:51:11] + Processing work unit
[20:51:11] Core required: FahCore_11.exe
[20:51:11] Core found.
[20:51:11] Working on queue slot 09 [June 12 20:51:11 UTC]
[20:51:11] + Working ...
[20:51:11] - Calling '.\FahCore_11.exe -dir work/ -suffix 09 -nice 19 -priority 96 -nocpulock -checkpoint 15 

-service -verbose -lifeline 1608 -version 641'

[20:51:11] 
[20:51:11] *------------------------------*
[20:51:11] Folding@Home GPU Core
[20:51:11] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[20:51:11] 
[20:51:11] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[20:51:11] Build host: amoeba
[20:51:11] Board Type: Nvidia
[20:51:11] Core      : 
[20:51:11] Preparing to commence simulation
[20:51:11] - Looking at optimizations...
[20:51:11] DeleteFrameFiles: successfully deleted file=work/wudata_09.ckp
[20:51:11] - Created dyn
[20:51:11] - Files status OK
[20:51:11] - Expanded 22987 -> 150831 (decompressed 656.1 percent)
[20:51:11] Called DecompressByteArray: compressed_data_size=22987 data_size=150831, decompressed_data_size=150831 

diff=0
[20:51:12] - Digital signature verified
[20:51:12] 
[20:51:12] Project: 10505 (Run 77, Clone 32, Gen 5)
[20:51:12] 
[20:51:12] Assembly optimizations on if available.
[20:51:12] Entering M.D.
[20:51:18] Tpr hash work/wudata_09.tpr:  2756390222 53604964 3070339681 2242771314 3530380191
[20:51:18] 
[20:51:18] Calling fah_main args: 14 usage=100
[20:51:18] 
[20:51:18] Working on Protein
[20:51:18] Client config found, loading data.
[20:51:18] mdrun_gpu returned 
[20:51:18] Going to send back what have done -- stepsTotalG=30000000
[20:51:18] Work fraction=0.0000 steps=30000000.
[20:51:18] Starting GUI Server
[20:51:22] logfile size=9591 infoLength=9591 edr=0 trr=25
[20:51:22] + Opened results file
[20:51:22] - Writing 10129 bytes of core data to disk...
[20:51:22] Decompress: Memory problem 2
[20:51:22] - Compressed data size (2089877597) exceeds limit. 
[20:51:22] DeleteFrameFiles: successfully deleted file=work/wudata_09.ckp
[20:51:22] 
[20:51:22] Folding@home Core Shutdown: EARLY_UNIT_END
[20:51:26] CoreStatus = 72 (114)
[20:51:26] Sending work to server
[20:51:26] Project: 10505 (Run 77, Clone 32, Gen 5)
[20:51:26] - Read packet limit of 540015616... Set to 524286976.


[20:51:26] + Attempting to send results [June 12 20:51:26 UTC]
[20:51:26] - Reading file work/wuresults_09.dat from core
[20:51:26]   (Read 512 bytes from disk)
[20:51:26] Gpu type=2 species=11.
[20:51:26] Connecting to http://171.67.108.21:8080/
[20:51:26] Posted data.
[20:51:26] Initial: 0000; Conversation time very short, giving reduced weight in bandwidth avg
[20:51:26] - Uploaded at ~3 kB/s
[20:51:26] - Averaged speed for that direction ~37 kB/s
[20:51:26] - Server reports problem with unit.
[20:51:26] Trying to send all finished work units
[20:51:26] + No unsent completed units remaining.
[20:51:26] - Preparing to get new work unit...
[20:51:26] Cleaning up work directory
[20:51:26] + Attempting to get work packet
[20:51:26] Passkey found
[20:51:26] - Will indicate memory of 2046 MB
[20:51:26] Gpu type=2 species=11.
[20:51:26] - Connecting to assignment server
[20:51:26] Connecting to http://assign-GPU.stanford.edu:8080/
[20:51:27] Posted data.
[20:51:27] Initial: 43AB; - Successful: assigned to (171.67.108.21).
[20:51:27] + News From Folding@Home: Welcome to Folding@Home
[20:51:27] Loaded queue successfully.
[20:51:27] Gpu type=2 species=11.
[20:51:27] Sent data
[20:51:27] Connecting to http://171.67.108.21:8080/
[20:51:28] Posted data.
[20:51:28] Initial: 0000; - Receiving payload (expected size: 23568)
[20:51:28] Conversation time very short, giving reduced weight in bandwidth avg
[20:51:28] - Downloaded at ~46 kB/s
[20:51:28] - Averaged speed for that direction ~42 kB/s
[20:51:28] + Received work.
[20:51:28] Trying to send all finished work units
[20:51:28] + No unsent completed units remaining.
[20:51:28] + Closed connections
[20:51:33] 
[20:51:33] + Processing work unit
[20:51:33] Core required: FahCore_11.exe
[20:51:33] Core found.
[20:51:33] Working on queue slot 00 [June 12 20:51:33 UTC]
[20:51:33] + Working ...
[20:51:33] - Calling '.\FahCore_11.exe -dir work/ -suffix 00 -nice 19 -priority 96 -nocpulock -checkpoint 15 

-service -verbose -lifeline 1608 -version 641'

[20:51:33] 
[20:51:33] *------------------------------*
[20:51:33] Folding@Home GPU Core
[20:51:33] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[20:51:33] 
[20:51:33] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[20:51:33] Build host: amoeba
[20:51:33] Board Type: Nvidia
[20:51:33] Core      : 
[20:51:33] Preparing to commence simulation
[20:51:33] - Looking at optimizations...
[20:51:33] DeleteFrameFiles: successfully deleted file=work/wudata_00.ckp
[20:51:33] - Created dyn
[20:51:33] - Files status OK
[20:51:33] - Expanded 23056 -> 149387 (decompressed 647.9 percent)
[20:51:33] Called DecompressByteArray: compressed_data_size=23056 data_size=149387, decompressed_data_size=149387 

diff=0
[20:51:33] - Digital signature verified
[20:51:33] 
[20:51:33] Project: 10505 (Run 78, Clone 32, Gen 5)
[20:51:33] 
[20:51:33] Assembly optimizations on if available.
[20:51:33] Entering M.D.
[20:51:39] Tpr hash work/wudata_00.tpr:  1756645554 505192081 2766852385 820681574 812067189
[20:51:39] 
[20:51:39] Calling fah_main args: 14 usage=100
[20:51:39] 
[20:51:39] Working on Protein
[20:51:39] Client config found, loading data.
[20:51:39] mdrun_gpu returned 
[20:51:39] Going to send back what have done -- stepsTotalG=30000000
[20:51:39] Work fraction=0.0000 steps=30000000.
[20:51:40] Starting GUI Server
[20:51:43] logfile size=9593 infoLength=9593 edr=0 trr=25
[20:51:43] + Opened results file
[20:51:43] - Writing 10131 bytes of core data to disk...
[20:51:44] Decompress: Memory problem 2
[20:51:44] - Compressed data size (2089877597) exceeds limit. 
[20:51:44] DeleteFrameFiles: successfully deleted file=work/wudata_00.ckp
[20:51:44] 
[20:51:44] Folding@home Core Shutdown: EARLY_UNIT_END
[20:51:47] CoreStatus = 72 (114)
[20:51:47] Sending work to server
[20:51:47] Project: 10505 (Run 78, Clone 32, Gen 5)
[20:51:47] - Read packet limit of 540015616... Set to 524286976.


[20:51:47] + Attempting to send results [June 12 20:51:47 UTC]
[20:51:47] - Reading file work/wuresults_00.dat from core
[20:51:47]   (Read 512 bytes from disk)
[20:51:47] Gpu type=2 species=11.
[20:51:47] Connecting to http://171.67.108.21:8080/
[20:51:47] Posted data.
[20:51:47] Initial: 0000; Conversation time very short, giving reduced weight in bandwidth avg
[20:51:47] - Uploaded at ~3 kB/s
[20:51:47] - Averaged speed for that direction ~33 kB/s
[20:51:47] - Server reports problem with unit.
[20:51:47] Trying to send all finished work units
[20:51:47] + No unsent completed units remaining.
[20:51:47] - Preparing to get new work unit...
[20:51:47] Cleaning up work directory
[20:51:47] + Attempting to get work packet
[20:51:47] Passkey found
[20:51:47] - Will indicate memory of 2046 MB
[20:51:47] Gpu type=2 species=11.
[20:51:47] - Connecting to assignment server
[20:51:47] Connecting to http://assign-GPU.stanford.edu:8080/
[20:51:48] Posted data.
[20:51:48] Initial: 43AB; - Successful: assigned to (171.67.108.21).
[20:51:48] + News From Folding@Home: Welcome to Folding@Home
[20:51:48] Loaded queue successfully.
[20:51:48] Gpu type=2 species=11.
[20:51:48] Sent data
[20:51:48] Connecting to http://171.67.108.21:8080/
[20:51:49] Posted data.
[20:51:49] Initial: 0000; - Receiving payload (expected size: 22600)
[20:51:49] Conversation time very short, giving reduced weight in bandwidth avg
[20:51:49] - Downloaded at ~44 kB/s
[20:51:49] - Averaged speed for that direction ~42 kB/s
[20:51:49] + Received work.
[20:51:49] Trying to send all finished work units
[20:51:49] + No unsent completed units remaining.
[20:51:49] + Closed connections
[20:51:54] 
[20:51:54] + Processing work unit
[20:51:54] Core required: FahCore_11.exe
[20:51:54] Core found.
[20:51:54] Working on queue slot 01 [June 12 20:51:54 UTC]
[20:51:54] + Working ...
[20:51:54] - Calling '.\FahCore_11.exe -dir work/ -suffix 01 -nice 19 -priority 96 -nocpulock -checkpoint 15 

-service -verbose -lifeline 1608 -version 641'

[20:51:54] 
[20:51:54] *------------------------------*
[20:51:54] Folding@Home GPU Core
[20:51:54] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[20:51:54] 
[20:51:54] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[20:51:54] Build host: amoeba
[20:51:54] Board Type: Nvidia
[20:51:54] Core      : 
[20:51:54] Preparing to commence simulation
[20:51:54] - Looking at optimizations...
[20:51:54] DeleteFrameFiles: successfully deleted file=work/wudata_01.ckp
[20:51:54] - Created dyn
[20:51:54] - Files status OK
[20:51:54] - Expanded 22088 -> 145731 (decompressed 659.7 percent)
[20:51:54] Called DecompressByteArray: compressed_data_size=22088 data_size=145731, decompressed_data_size=145731 

diff=0
[20:51:55] - Digital signature verified
[20:51:55] 
[20:51:55] Project: 10505 (Run 79, Clone 32, Gen 5)
[20:51:55] 
[20:51:55] Assembly optimizations on if available.
[20:51:55] Entering M.D.
[20:52:01] Tpr hash work/wudata_01.tpr:  2671375074 557246923 4207003573 882642501 1526140284
[20:52:01] 
[20:52:01] Calling fah_main args: 14 usage=100
[20:52:01] 
[20:52:01] Run: exception thrown during GuardedRun
[20:52:01] Run: exception thrown in GuardedRun -- Gromacs cannot continue further.
[20:52:01] Going to send back what have done -- stepsTotalG=0
[20:52:01] Work fraction=0.0000 steps=0.
[20:52:05] logfile size=4943 infoLength=4943 edr=0 trr=23
[20:52:05] + Opened results file
[20:52:05] - Writing 5479 bytes of core data to disk...
[20:52:05] Done: 4967 -> 1841 (compressed to 37.0 percent)
[20:52:05]   ... Done.
[20:52:05] DeleteFrameFiles: successfully deleted file=work/wudata_01.ckp
[20:52:05] 
[20:52:05] Folding@home Core Shutdown: UNSTABLE_MACHINE
[20:52:08] CoreStatus = 7A (122)
[20:52:08] Sending work to server
[20:52:08] Project: 10505 (Run 79, Clone 32, Gen 5)
[20:52:08] - Read packet limit of 540015616... Set to 524286976.


[20:52:08] + Attempting to send results [June 12 20:52:08 UTC]
[20:52:08] - Reading file work/wuresults_01.dat from core
[20:52:08]   (Read 2353 bytes from disk)
[20:52:08] Gpu type=2 species=11.
[20:52:08] Connecting to http://171.67.108.21:8080/
[20:52:09] Posted data.
[20:52:09] Initial: 0000; - Uploaded at ~3 kB/s
[20:52:09] - Averaged speed for that direction ~27 kB/s
[20:52:09] + Results successfully sent
[20:52:09] Thank you for your contribution to Folding@Home.
[20:52:13] Trying to send all finished work units
[20:52:13] + No unsent completed units remaining.
[20:52:13] - Preparing to get new work unit...
[20:52:13] Cleaning up work directory
[20:52:13] + Attempting to get work packet
[20:52:13] Passkey found
[20:52:13] - Will indicate memory of 2046 MB
[20:52:13] Gpu type=2 species=11.
[20:52:13] - Connecting to assignment server
[20:52:13] Connecting to http://assign-GPU.stanford.edu:8080/
[20:52:14] Posted data.
[20:52:14] Initial: 43AB; - Successful: assigned to (171.67.108.21).
[20:52:14] + News From Folding@Home: Welcome to Folding@Home
[20:52:14] Loaded queue successfully.
[20:52:14] Gpu type=2 species=11.
[20:52:14] Sent data
[20:52:14] Connecting to http://171.67.108.21:8080/
[20:52:14] Posted data.
[20:52:14] Initial: 0000; - Receiving payload (expected size: 23085)
[20:52:14] Conversation time very short, giving reduced weight in bandwidth avg
[20:52:14] - Downloaded at ~45 kB/s
[20:52:14] - Averaged speed for that direction ~42 kB/s
[20:52:14] + Received work.
[20:52:14] Trying to send all finished work units
[20:52:14] + No unsent completed units remaining.
[20:52:14] + Closed connections
[20:52:19] 
[20:52:19] + Processing work unit
[20:52:19] Core required: FahCore_11.exe
[20:52:19] Core found.
[20:52:19] Working on queue slot 02 [June 12 20:52:19 UTC]
[20:52:19] + Working ...
[20:52:19] - Calling '.\FahCore_11.exe -dir work/ -suffix 02 -nice 19 -priority 96 -nocpulock -checkpoint 15 

-service -verbose -lifeline 1608 -version 641'

[20:52:20] 
[20:52:20] *------------------------------*
[20:52:20] Folding@Home GPU Core
[20:52:20] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[20:52:20] 
[20:52:20] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[20:52:20] Build host: amoeba
[20:52:20] Board Type: Nvidia
[20:52:20] Core      : 
[20:52:20] Preparing to commence simulation
[20:52:20] - Looking at optimizations...
[20:52:20] DeleteFrameFiles: successfully deleted file=work/wudata_02.ckp
[20:52:20] - Created dyn
[20:52:20] - Files status OK
[20:52:20] - Expanded 22573 -> 147695 (decompressed 654.2 percent)
[20:52:20] Called DecompressByteArray: compressed_data_size=22573 data_size=147695, decompressed_data_size=147695 

diff=0
[20:52:20] - Digital signature verified
[20:52:20] 
[20:52:20] Project: 10505 (Run 80, Clone 32, Gen 5)
[20:52:20] 
[20:52:20] Assembly optimizations on if available.
[20:52:20] Entering M.D.
[20:52:26] Tpr hash work/wudata_02.tpr:  582071458 2927201036 2084967429 639347892 1719258630
[20:52:26] 
[20:52:26] Calling fah_main args: 14 usage=100
[20:52:26] 
[20:52:26] Working on Protein
[20:52:26] Run: exception thrown during GuardedRun
[20:52:26] Run: exception thrown in GuardedRun -- Gromacs cannot continue further.
[20:52:26] Going to send back what have done -- stepsTotalG=0
[20:52:26] Work fraction=0.0000 steps=0.
[20:52:30] logfile size=9151 infoLength=9151 edr=0 trr=23
[20:52:30] + Opened results file
[20:52:30] - Writing 9687 bytes of core data to disk...
[20:52:30] Decompress: Memory problem 2
[20:52:30] - Compressed data size (2089877597) exceeds limit. 
[20:52:30] DeleteFrameFiles: successfully deleted file=work/wudata_02.ckp
[20:52:30] 
[20:52:30] Folding@home Core Shutdown: UNSTABLE_MACHINE
[20:52:34] CoreStatus = 7A (122)
[20:52:34] Sending work to server
[20:52:34] Project: 10505 (Run 80, Clone 32, Gen 5)
[20:52:34] - Read packet limit of 540015616... Set to 524286976.


[20:52:34] + Attempting to send results [June 12 20:52:34 UTC]
[20:52:34] - Reading file work/wuresults_02.dat from core
[20:52:34]   (Read 512 bytes from disk)
[20:52:34] Gpu type=2 species=11.
[20:52:34] Connecting to http://171.67.108.21:8080/
[20:52:34] Posted data.
[20:52:34] Initial: 0000; Conversation time very short, giving reduced weight in bandwidth avg
[20:52:34] - Uploaded at ~3 kB/s
[20:52:34] - Averaged speed for that direction ~24 kB/s
[20:52:34] - Server reports problem with unit.
[20:52:34] Trying to send all finished work units
[20:52:34] + No unsent completed units remaining.
[20:52:34] - Preparing to get new work unit...
[20:52:34] Cleaning up work directory
[20:52:34] + Attempting to get work packet
[20:52:34] Passkey found
[20:52:34] - Will indicate memory of 2046 MB
[20:52:34] Gpu type=2 species=11.
[20:52:34] - Connecting to assignment server
[20:52:34] Connecting to http://assign-GPU.stanford.edu:8080/
[20:52:35] Posted data.
[20:52:35] Initial: 43AB; - Successful: assigned to (171.67.108.21).
[20:52:35] + News From Folding@Home: Welcome to Folding@Home
[20:52:35] Loaded queue successfully.
[20:52:35] Gpu type=2 species=11.
[20:52:35] Sent data
[20:52:35] Connecting to http://171.67.108.21:8080/
[20:52:36] Posted data.
[20:52:36] Initial: 0000; - Receiving payload (expected size: 22406)
[20:52:36] Conversation time very short, giving reduced weight in bandwidth avg
[20:52:36] - Downloaded at ~43 kB/s
[20:52:36] - Averaged speed for that direction ~42 kB/s
[20:52:36] + Received work.
[20:52:36] Trying to send all finished work units
[20:52:36] + No unsent completed units remaining.
[20:52:36] + Closed connections
[20:52:41] 
[20:52:41] + Processing work unit
[20:52:41] Core required: FahCore_11.exe
[20:52:41] Core found.
[20:52:41] Working on queue slot 03 [June 12 20:52:41 UTC]
[20:52:41] + Working ...
[20:52:41] - Calling '.\FahCore_11.exe -dir work/ -suffix 03 -nice 19 -priority 96 -nocpulock -checkpoint 15 

-service -verbose -lifeline 1608 -version 641'

[20:52:41] 
[20:52:41] *------------------------------*
[20:52:41] Folding@Home GPU Core
[20:52:41] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[20:52:41] 
[20:52:41] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[20:52:41] Build host: amoeba
[20:52:41] Board Type: Nvidia
[20:52:41] Core      : 
[20:52:41] Preparing to commence simulation
[20:52:41] - Looking at optimizations...
[20:52:41] DeleteFrameFiles: successfully deleted file=work/wudata_03.ckp
[20:52:41] - Created dyn
[20:52:41] - Files status OK
[20:52:41] - Expanded 21894 -> 144751 (decompressed 661.1 percent)
[20:52:41] Called DecompressByteArray: compressed_data_size=21894 data_size=144751, decompressed_data_size=144751 

diff=0
[20:52:41] - Digital signature verified
[20:52:41] 
[20:52:41] Project: 10505 (Run 81, Clone 32, Gen 5)
[20:52:41] 
[20:52:41] Assembly optimizations on if available.
[20:52:41] Entering M.D.
[20:52:47] Tpr hash work/wudata_03.tpr:  4141331038 2640879855 451980419 1471201792 3550793233
[20:52:47] 
[20:52:47] Calling fah_main args: 14 usage=100
[20:52:47] 
[20:52:47] Working on Protein
[20:52:47] Run: exception thrown during GuardedRun
[20:52:47] Run: exception thrown in GuardedRun -- Gromacs cannot continue further.
[20:52:47] Going to send back what have done -- stepsTotalG=0
[20:52:47] Work fraction=0.0000 steps=0.
[20:52:51] logfile size=9150 infoLength=9150 edr=0 trr=23
[20:52:51] + Opened results file
[20:52:51] - Writing 9686 bytes of core data to disk...
[20:52:51] Decompress: Memory problem 2
[20:52:51] - Compressed data size (2089877597) exceeds limit. 
[20:52:51] DeleteFrameFiles: successfully deleted file=work/wudata_03.ckp
[20:52:52] 
[20:52:52] Folding@home Core Shutdown: UNSTABLE_MACHINE
[20:52:55] CoreStatus = 7A (122)
[20:52:55] Sending work to server
[20:52:55] Project: 10505 (Run 81, Clone 32, Gen 5)
[20:52:55] - Read packet limit of 540015616... Set to 524286976.


[20:52:55] + Attempting to send results [June 12 20:52:55 UTC]
[20:52:55] - Reading file work/wuresults_03.dat from core
[20:52:55]   (Read 512 bytes from disk)
[20:52:55] Gpu type=2 species=11.
[20:52:55] Connecting to http://171.67.108.21:8080/
[20:52:55] Posted data.
[20:52:55] Initial: 0000; Conversation time very short, giving reduced weight in bandwidth avg
[20:52:55] - Uploaded at ~3 kB/s
[20:52:55] - Averaged speed for that direction ~22 kB/s
[20:52:55] - Server reports problem with unit.
[20:52:55] Trying to send all finished work units
[20:52:55] + No unsent completed units remaining.
[20:52:55] - Preparing to get new work unit...
[20:52:55] Cleaning up work directory
[20:52:55] + Attempting to get work packet
[20:52:55] Passkey found
[20:52:55] - Will indicate memory of 2046 MB
[20:52:56] Gpu type=2 species=11.
[20:52:56] - Connecting to assignment server
[20:52:56] Connecting to http://assign-GPU.stanford.edu:8080/
[20:52:56] Posted data.
[20:52:56] Initial: 43AB; - Successful: assigned to (171.67.108.21).
[20:52:56] + News From Folding@Home: Welcome to Folding@Home
[20:52:56] Loaded queue successfully.
[20:52:56] Gpu type=2 species=11.
[20:52:56] Sent data
[20:52:56] Connecting to http://171.67.108.21:8080/
[20:52:57] Posted data.
[20:52:57] Initial: 0000; - Receiving payload (expected size: 20915)
[20:52:57] Conversation time very short, giving reduced weight in bandwidth avg
[20:52:57] - Downloaded at ~40 kB/s
[20:52:57] - Averaged speed for that direction ~42 kB/s
[20:52:57] + Received work.
[20:52:57] Trying to send all finished work units
[20:52:57] + No unsent completed units remaining.
[20:52:57] + Closed connections
[20:53:02] 
[20:53:02] + Processing work unit
[20:53:02] Core required: FahCore_11.exe
[20:53:02] Core found.
[20:53:02] Working on queue slot 04 [June 12 20:53:02 UTC]
[20:53:02] + Working ...
[20:53:02] - Calling '.\FahCore_11.exe -dir work/ -suffix 04 -nice 19 -priority 96 -nocpulock -checkpoint 15 

-service -verbose -lifeline 1608 -version 641'

[20:53:02] 
[20:53:02] *------------------------------*
[20:53:02] Folding@Home GPU Core
[20:53:02] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[20:53:02] 
[20:53:02] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[20:53:02] Build host: amoeba
[20:53:02] Board Type: Nvidia
[20:53:02] Core      : 
[20:53:02] Preparing to commence simulation
[20:53:02] - Looking at optimizations...
[20:53:02] DeleteFrameFiles: successfully deleted file=work/wudata_04.ckp
[20:53:02] - Created dyn
[20:53:02] - Files status OK
[20:53:02] - Expanded 20403 -> 137455 (decompressed 673.6 percent)
[20:53:02] Called DecompressByteArray: compressed_data_size=20403 data_size=137455, decompressed_data_size=137455 

diff=0
[20:53:02] - Digital signature verified
[20:53:02] 
[20:53:02] Project: 10505 (Run 82, Clone 32, Gen 5)
[20:53:02] 
[20:53:02] Assembly optimizations on if available.
[20:53:02] Entering M.D.
[20:53:08] Tpr hash work/wudata_04.tpr:  384082013 2659374219 2926612150 2585667547 2344854909
[20:53:08] 
[20:53:08] Calling fah_main args: 14 usage=100
[20:53:08] 
[20:53:08] Run: exception thrown during GuardedRun
[20:53:08] Run: exception thrown in GuardedRun -- Gromacs cannot continue further.
[20:53:08] Going to send back what have done -- stepsTotalG=0
[20:53:08] Work fraction=0.0000 steps=0.
[20:53:12] logfile size=4943 infoLength=4943 edr=0 trr=23
[20:53:12] + Opened results file
[20:53:12] - Writing 5479 bytes of core data to disk...
[20:53:12] Decompress: Memory problem 2
[20:53:12] - Compressed data size (2089877597) exceeds limit. 
[20:53:12] DeleteFrameFiles: successfully deleted file=work/wudata_04.ckp
[20:53:12] 
[20:53:12] Folding@home Core Shutdown: UNSTABLE_MACHINE
[20:53:16] CoreStatus = 7A (122)
[20:53:16] Sending work to server
[20:53:16] Project: 10505 (Run 82, Clone 32, Gen 5)
[20:53:16] - Read packet limit of 540015616... Set to 524286976.


[20:53:16] + Attempting to send results [June 12 20:53:16 UTC]
[20:53:16] - Reading file work/wuresults_04.dat from core
[20:53:16]   (Read 512 bytes from disk)
[20:53:16] Gpu type=2 species=11.
[20:53:16] Connecting to http://171.67.108.21:8080/
[20:53:17] Posted data.
[20:53:17] Initial: 0000; - Uploaded at ~1 kB/s
[20:53:17] - Averaged speed for that direction ~18 kB/s
[20:53:17] - Server reports problem with unit.
[20:53:17] Trying to send all finished work units
[20:53:17] + No unsent completed units remaining.
[20:53:17] - Preparing to get new work unit...
[20:53:17] Cleaning up work directory
[20:53:17] + Attempting to get work packet
[20:53:17] Passkey found
[20:53:17] - Will indicate memory of 2046 MB
[20:53:17] Gpu type=2 species=11.
[20:53:17] - Connecting to assignment server
[20:53:17] Connecting to http://assign-GPU.stanford.edu:8080/
[20:53:17] Posted data.
[20:53:17] Initial: 43AB; - Successful: assigned to (171.67.108.21).
[20:53:17] + News From Folding@Home: Welcome to Folding@Home
[20:53:18] Loaded queue successfully.
[20:53:18] Gpu type=2 species=11.
[20:53:18] Sent data
[20:53:18] Connecting to http://171.67.108.21:8080/
[20:53:18] Posted data.
[20:53:18] Initial: 0000; - Receiving payload (expected size: 21481)
[20:53:18] Conversation time very short, giving reduced weight in bandwidth avg
[20:53:18] - Downloaded at ~41 kB/s
[20:53:18] - Averaged speed for that direction ~42 kB/s
[20:53:18] + Received work.
[20:53:18] Trying to send all finished work units
[20:53:18] + No unsent completed units remaining.
[20:53:18] + Closed connections
[20:53:23] 
[20:53:23] + Processing work unit
[20:53:23] Core required: FahCore_11.exe
[20:53:23] Core found.
[20:53:23] Working on queue slot 05 [June 12 20:53:23 UTC]
[20:53:23] + Working ...
[20:53:23] - Calling '.\FahCore_11.exe -dir work/ -suffix 05 -nice 19 -priority 96 -nocpulock -checkpoint 15 

-service -verbose -lifeline 1608 -version 641'

[20:53:23] 
[20:53:23] *------------------------------*
[20:53:23] Folding@Home GPU Core
[20:53:23] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[20:53:23] 
[20:53:23] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[20:53:23] Build host: amoeba
[20:53:23] Board Type: Nvidia
[20:53:23] Core      : 
[20:53:23] Preparing to commence simulation
[20:53:23] - Looking at optimizations...
[20:53:23] DeleteFrameFiles: successfully deleted file=work/wudata_05.ckp
[20:53:23] - Created dyn
[20:53:23] - Files status OK
[20:53:23] - Expanded 20969 -> 139911 (decompressed 667.2 percent)
[20:53:23] Called DecompressByteArray: compressed_data_size=20969 data_size=139911, decompressed_data_size=139911 

diff=0
[20:53:23] - Digital signature verified
[20:53:23] 
[20:53:23] Project: 10505 (Run 83, Clone 32, Gen 5)
[20:53:23] 
[20:53:23] Assembly optimizations on if available.
[20:53:23] Entering M.D.
[20:53:29] Tpr hash work/wudata_05.tpr:  2301567363 2398545030 2528224608 924774523 2546019541
[20:53:29] 
[20:53:29] Calling fah_main args: 14 usage=100
[20:53:29] 
[20:53:30] Run: exception thrown during GuardedRun
[20:53:30] Run: exception thrown in GuardedRun -- Gromacs cannot continue further.
[20:53:30] Going to send back what have done -- stepsTotalG=0
[20:53:30] Work fraction=0.0000 steps=0.
[20:53:34] logfile size=4943 infoLength=4943 edr=0 trr=23
[20:53:34] + Opened results file
[20:53:34] - Writing 5479 bytes of core data to disk...
[20:53:34] Decompress: Memory problem 2
[20:53:34] - Compressed data size (2089877597) exceeds limit. 
[20:53:34] DeleteFrameFiles: successfully deleted file=work/wudata_05.ckp
[20:53:34] 
[20:53:34] Folding@home Core Shutdown: UNSTABLE_MACHINE
[20:53:37] CoreStatus = 7A (122)
[20:53:37] Sending work to server
[20:53:37] Project: 10505 (Run 83, Clone 32, Gen 5)
[20:53:37] - Read packet limit of 540015616... Set to 524286976.


[20:53:37] + Attempting to send results [June 12 20:53:37 UTC]
[20:53:37] - Reading file work/wuresults_05.dat from core
[20:53:37]   (Read 512 bytes from disk)
[20:53:37] Gpu type=2 species=11.
[20:53:37] Connecting to http://171.67.108.21:8080/
[20:53:38] Posted data.
[20:53:38] Initial: 0000; - Uploaded at ~1 kB/s
[20:53:38] - Averaged speed for that direction ~14 kB/s
[20:53:38] - Server reports problem with unit.
[20:53:38] EUE limit exceeded. Pausing 24 hours.
[02:28:37] - Autosending finished units... [June 13 02:28:37 UTC]
[02:28:37] Trying to send all finished work units
[02:28:37] + No unsent completed units remaining.
[02:28:37] - Autosend completed
[08:28:38] - Autosending finished units... [June 13 08:28:38 UTC]
[08:28:38] Trying to send all finished work units
[08:28:38] + No unsent completed units remaining.
[08:28:38] - Autosend completed
Folding since 1 WU=1 point
ImageImage
toTOW
Site Moderator
Posts: 6296
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:

Re: Odd fail pattern 10505 (Run 1xx, Clone 33, Gen 5) +anoth

Post by toTOW »

I didn't notice any issues with this project on my two 9800 GTX+ ... but I might not had WUs from the same clone.

I checked the WUs you listed, and they were all completed successfully by someone else.

But the fun thing is that 95% of the WU you listed were completed successfully ... by the same user ...
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
Post Reply