WU 9016 stuck at "Sending"

Moderators: Site Moderators, FAHC Science Team

Post Reply
Donk_Enterprises
Posts: 4
Joined: Sat May 02, 2015 5:13 am

WU 9016 stuck at "Sending"

Post by Donk_Enterprises »

PRCG 9016 (405, 3, 155) shows 100% Progress and has been in "Send" status for 2 days or so. Is there a way to give it a kick to get it to retry sending? It looks like the client has moved on to crunching other WUs since then, I have 2 active threads as usual (one for GPU and one for CPU) in addition to the one that's stuck in "send" mode.

Work log shows some errors that may have a bearing on this:

Code: Select all

******************************* Date: 2015-04-28 *******************************
17:56:51:WARNING:WU01:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
17:58:07:ERROR:WU01:FS00:Exception: 10002: Received short response, expected 512 bytes, got 0
17:58:09:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
17:58:10:WARNING:WU01:FS00:Exception: Failed to send results to work server: Failed to connect to 171.64.65.124:80: No connection could be made because the target machine actively refused it.
But I think it may have been sent ok from what this log says

Code: Select all

******************************* Date: 2015-04-28 *******************************
01:48:32:WU01:FS00:Connecting to 171.67.108.200:8080
01:48:32:WU01:FS00:Assigned to work server 171.64.65.124
01:48:32:WU01:FS00:Requesting new work unit for slot 00: RUNNING cpu:3 from 171.64.65.124
01:48:32:WU01:FS00:Connecting to 171.64.65.124:8080
01:48:33:WU01:FS00:Downloading 807.58KiB
01:48:33:WU01:FS00:Download complete
01:48:34:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:9016 run:405 clone:3 gen:155 core:0xa4 unit:0x000000b4664f2de45491d98025b640f8
01:53:32:WU01:FS00:Starting
01:53:32:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/John/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 704 -lifeline 5552 -checkpoint 15 -np 3
01:53:33:WU01:FS00:Started FahCore on PID 3680
01:53:33:WU01:FS00:Core PID:1580
01:53:33:WU01:FS00:FahCore 0xa4 started
01:53:33:WU01:FS00:0xa4:
01:53:33:WU01:FS00:0xa4:*------------------------------*
01:53:33:WU01:FS00:0xa4:Folding@Home Gromacs GB Core
01:53:33:WU01:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
01:53:33:WU01:FS00:0xa4:
01:53:33:WU01:FS00:0xa4:Preparing to commence simulation
01:53:33:WU01:FS00:0xa4:- Looking at optimizations...
01:53:33:WU01:FS00:0xa4:- Created dyn
01:53:33:WU01:FS00:0xa4:- Files status OK
01:53:33:WU01:FS00:0xa4:- Expanded 826451 -> 1397548 (decompressed 169.1 percent)
01:53:33:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=826451 data_size=1397548, decompressed_data_size=1397548 diff=0
01:53:33:WU01:FS00:0xa4:- Digital signature verified
01:53:33:WU01:FS00:0xa4:
01:53:33:WU01:FS00:0xa4:Project: 9016 (Run 405, Clone 3, Gen 155)
01:53:33:WU01:FS00:0xa4:
01:53:33:WU01:FS00:0xa4:Assembly optimizations on if available.
01:53:33:WU01:FS00:0xa4:Entering M.D.
01:53:39:WU01:FS00:0xa4:Mapping NT from 3 to 3 
01:53:40:WU01:FS00:0xa4:Completed 0 out of 250000 steps  (0%)
01:57:20:WU01:FS00:0xa4:Completed 2500 out of 250000 steps  (1%)
02:00:59:WU01:FS00:0xa4:Completed 5000 out of 250000 steps  (2%)
02:04:36:WU01:FS00:0xa4:Completed 7500 out of 250000 steps  (3%)
02:08:15:WU01:FS00:0xa4:Completed 10000 out of 250000 steps  (4%)
02:11:54:WU01:FS00:0xa4:Completed 12500 out of 250000 steps  (5%)
02:15:33:WU01:FS00:0xa4:Completed 15000 out of 250000 steps  (6%)
02:19:12:WU01:FS00:0xa4:Completed 17500 out of 250000 steps  (7%)
02:22:50:WU01:FS00:0xa4:Completed 20000 out of 250000 steps  (8%)
02:26:30:WU01:FS00:0xa4:Completed 22500 out of 250000 steps  (9%)
02:30:08:WU01:FS00:0xa4:Completed 25000 out of 250000 steps  (10%)
02:33:47:WU01:FS00:0xa4:Completed 27500 out of 250000 steps  (11%)
02:37:27:WU01:FS00:0xa4:Completed 30000 out of 250000 steps  (12%)
02:41:07:WU01:FS00:0xa4:Completed 32500 out of 250000 steps  (13%)
02:44:47:WU01:FS00:0xa4:Completed 35000 out of 250000 steps  (14%)
02:48:28:WU01:FS00:0xa4:Completed 37500 out of 250000 steps  (15%)
02:52:08:WU01:FS00:0xa4:Completed 40000 out of 250000 steps  (16%)
02:55:49:WU01:FS00:0xa4:Completed 42500 out of 250000 steps  (17%)
02:59:28:WU01:FS00:0xa4:Completed 45000 out of 250000 steps  (18%)
03:03:07:WU01:FS00:0xa4:Completed 47500 out of 250000 steps  (19%)
03:06:47:WU01:FS00:0xa4:Completed 50000 out of 250000 steps  (20%)
03:10:27:WU01:FS00:0xa4:Completed 52500 out of 250000 steps  (21%)
03:14:06:WU01:FS00:0xa4:Completed 55000 out of 250000 steps  (22%)
03:17:46:WU01:FS00:0xa4:Completed 57500 out of 250000 steps  (23%)
03:21:26:WU01:FS00:0xa4:Completed 60000 out of 250000 steps  (24%)
03:25:05:WU01:FS00:0xa4:Completed 62500 out of 250000 steps  (25%)
03:28:46:WU01:FS00:0xa4:Completed 65000 out of 250000 steps  (26%)
03:32:26:WU01:FS00:0xa4:Completed 67500 out of 250000 steps  (27%)
03:36:05:WU01:FS00:0xa4:Completed 70000 out of 250000 steps  (28%)
03:39:46:WU01:FS00:0xa4:Completed 72500 out of 250000 steps  (29%)
03:43:26:WU01:FS00:0xa4:Completed 75000 out of 250000 steps  (30%)
03:47:06:WU01:FS00:0xa4:Completed 77500 out of 250000 steps  (31%)
03:50:44:WU01:FS00:0xa4:Completed 80000 out of 250000 steps  (32%)
03:54:24:WU01:FS00:0xa4:Completed 82500 out of 250000 steps  (33%)
03:58:03:WU01:FS00:0xa4:Completed 85000 out of 250000 steps  (34%)
04:01:42:WU01:FS00:0xa4:Completed 87500 out of 250000 steps  (35%)
04:05:20:WU01:FS00:0xa4:Completed 90000 out of 250000 steps  (36%)
04:08:59:WU01:FS00:0xa4:Completed 92500 out of 250000 steps  (37%)
04:12:38:WU01:FS00:0xa4:Completed 95000 out of 250000 steps  (38%)
04:16:15:WU01:FS00:0xa4:Completed 97500 out of 250000 steps  (39%)
04:19:53:WU01:FS00:0xa4:Completed 100000 out of 250000 steps  (40%)
04:23:31:WU01:FS00:0xa4:Completed 102500 out of 250000 steps  (41%)
04:27:10:WU01:FS00:0xa4:Completed 105000 out of 250000 steps  (42%)
04:30:49:WU01:FS00:0xa4:Completed 107500 out of 250000 steps  (43%)
04:34:28:WU01:FS00:0xa4:Completed 110000 out of 250000 steps  (44%)
04:38:06:WU01:FS00:0xa4:Completed 112500 out of 250000 steps  (45%)
04:41:45:WU01:FS00:0xa4:Completed 115000 out of 250000 steps  (46%)
04:45:25:WU01:FS00:0xa4:Completed 117500 out of 250000 steps  (47%)
04:49:06:WU01:FS00:0xa4:Completed 120000 out of 250000 steps  (48%)
******************************* Date: 2015-04-29 *******************************
04:52:46:WU01:FS00:0xa4:Completed 122500 out of 250000 steps  (49%)
04:56:37:WU01:FS00:0xa4:Completed 125000 out of 250000 steps  (50%)
05:00:32:WU01:FS00:0xa4:Completed 127500 out of 250000 steps  (51%)
05:04:58:WU01:FS00:0xa4:Completed 130000 out of 250000 steps  (52%)
05:08:56:WU01:FS00:0xa4:Completed 132500 out of 250000 steps  (53%)
05:12:48:WU01:FS00:0xa4:Completed 135000 out of 250000 steps  (54%)
05:16:41:WU01:FS00:0xa4:Completed 137500 out of 250000 steps  (55%)
05:20:34:WU01:FS00:0xa4:Completed 140000 out of 250000 steps  (56%)
05:24:27:WU01:FS00:0xa4:Completed 142500 out of 250000 steps  (57%)
05:28:25:WU01:FS00:0xa4:Completed 145000 out of 250000 steps  (58%)
05:32:18:WU01:FS00:0xa4:Completed 147500 out of 250000 steps  (59%)
05:36:15:WU01:FS00:0xa4:Completed 150000 out of 250000 steps  (60%)
05:40:11:WU01:FS00:0xa4:Completed 152500 out of 250000 steps  (61%)
05:44:05:WU01:FS00:0xa4:Completed 155000 out of 250000 steps  (62%)
05:48:01:WU01:FS00:0xa4:Completed 157500 out of 250000 steps  (63%)
05:51:56:WU01:FS00:0xa4:Completed 160000 out of 250000 steps  (64%)
05:55:52:WU01:FS00:0xa4:Completed 162500 out of 250000 steps  (65%)
05:59:49:WU01:FS00:0xa4:Completed 165000 out of 250000 steps  (66%)
06:03:44:WU01:FS00:0xa4:Completed 167500 out of 250000 steps  (67%)
06:07:40:WU01:FS00:0xa4:Completed 170000 out of 250000 steps  (68%)
06:11:36:WU01:FS00:0xa4:Completed 172500 out of 250000 steps  (69%)
06:15:31:WU01:FS00:0xa4:Completed 175000 out of 250000 steps  (70%)
06:19:25:WU01:FS00:0xa4:Completed 177500 out of 250000 steps  (71%)
06:23:21:WU01:FS00:0xa4:Completed 180000 out of 250000 steps  (72%)
06:27:16:WU01:FS00:0xa4:Completed 182500 out of 250000 steps  (73%)
06:31:11:WU01:FS00:0xa4:Completed 185000 out of 250000 steps  (74%)
06:35:07:WU01:FS00:0xa4:Completed 187500 out of 250000 steps  (75%)
06:39:04:WU01:FS00:0xa4:Completed 190000 out of 250000 steps  (76%)
06:43:00:WU01:FS00:0xa4:Completed 192500 out of 250000 steps  (77%)
06:46:57:WU01:FS00:0xa4:Completed 195000 out of 250000 steps  (78%)
06:50:53:WU01:FS00:0xa4:Completed 197500 out of 250000 steps  (79%)
06:54:50:WU01:FS00:0xa4:Completed 200000 out of 250000 steps  (80%)
06:58:48:WU01:FS00:0xa4:Completed 202500 out of 250000 steps  (81%)
07:02:48:WU01:FS00:0xa4:Completed 205000 out of 250000 steps  (82%)
07:06:48:WU01:FS00:0xa4:Completed 207500 out of 250000 steps  (83%)
07:10:46:WU01:FS00:0xa4:Completed 210000 out of 250000 steps  (84%)
07:14:41:WU01:FS00:0xa4:Completed 212500 out of 250000 steps  (85%)
07:18:36:WU01:FS00:0xa4:Completed 215000 out of 250000 steps  (86%)
07:22:30:WU01:FS00:0xa4:Completed 217500 out of 250000 steps  (87%)
07:26:23:WU01:FS00:0xa4:Completed 220000 out of 250000 steps  (88%)
07:30:54:WU01:FS00:0xa4:Completed 222500 out of 250000 steps  (89%)
07:36:22:WU01:FS00:0xa4:Completed 225000 out of 250000 steps  (90%)
07:41:54:WU01:FS00:0xa4:Completed 227500 out of 250000 steps  (91%)
07:45:48:WU01:FS00:0xa4:Completed 230000 out of 250000 steps  (92%)
07:50:00:WU01:FS00:0xa4:Completed 232500 out of 250000 steps  (93%)
07:54:24:WU01:FS00:0xa4:Completed 235000 out of 250000 steps  (94%)
07:58:30:WU01:FS00:0xa4:Completed 237500 out of 250000 steps  (95%)
08:02:27:WU01:FS00:0xa4:Completed 240000 out of 250000 steps  (96%)
08:06:26:WU01:FS00:0xa4:Completed 242500 out of 250000 steps  (97%)
08:10:34:WU01:FS00:0xa4:Completed 245000 out of 250000 steps  (98%)
08:14:38:WU01:FS00:0xa4:Completed 247500 out of 250000 steps  (99%)
08:19:27:WU01:FS00:0xa4:Completed 250000 out of 250000 steps  (100%)
08:19:28:WU01:FS00:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
08:19:38:WU01:FS00:0xa4:
08:19:38:WU01:FS00:0xa4:Finished Work Unit:
08:19:38:WU01:FS00:0xa4:- Reading up to 811488 from "01/wudata_01.trr": Read 811488
08:19:38:WU01:FS00:0xa4:trr file hash check passed.
08:19:38:WU01:FS00:0xa4:- Reading up to 746380 from "01/wudata_01.xtc": Read 746380
08:19:38:WU01:FS00:0xa4:xtc file hash check passed.
08:19:38:WU01:FS00:0xa4:edr file hash check passed.
08:19:38:WU01:FS00:0xa4:logfile size: 24002
08:19:38:WU01:FS00:0xa4:Leaving Run
08:19:43:WU01:FS00:0xa4:- Writing 1584358 bytes of core data to disk...
08:19:44:WU01:FS00:0xa4:Done: 1583846 -> 1538190 (compressed to 97.1 percent)
08:19:44:WU01:FS00:0xa4:  ... Done.
08:19:45:WU01:FS00:0xa4:- Shutting down core
08:19:45:WU01:FS00:0xa4:
08:19:45:WU01:FS00:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
08:19:46:WU01:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
08:19:46:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:9016 run:405 clone:3 gen:155 core:0xa4 unit:0x000000b4664f2de45491d98025b640f8
08:19:46:WU01:FS00:Uploading 1.47MiB to 171.64.65.124
08:19:46:WU01:FS00:Connecting to 171.64.65.124:8080
Mod edit: Added Code tags from the posting Full Editor to log extracts.
Joe_H
Site Admin
Posts: 7870
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: WU 9016 stuck at "Sending"

Post by Joe_H »

Welcome to the folding forum.

There are a few possible problems, but there is not enough of your log to differentiate between them. The first error message you show, "Received short response" usually comes from the connection between the server and your system's FAHClient process being blocked. That could be from something at your ISP or anti-malware software or firewall on your system or network.

Being followed by an "actively refused" message could point to a temporary problem on the server's end. We would need to see more of the log from before and after those messages to see if that is the case. In any case, has any work after this WU been successfully uploaded?

As for the second section of your log, the end only shows that the WU was finished and ready for upload. There should be messages after that showing the progress of the upload and then if accepted the estimated points to be credited. If the log does show that, please post that portion of the log showing that. I did check, this WU has not been received and entered into the stats database.

Finally, an important part of your log that is very useful in troubleshooting a problem is the first 100 or so lines of it which shows client version info, system info, and the folding configuration being used. Could you also post that?
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Donk_Enterprises
Posts: 4
Joined: Sat May 02, 2015 5:13 am

Re: WU 9016 stuck at "Sending"

Post by Donk_Enterprises »

Here's the log for slot 00 unit 01.

Code: Select all

*********************** Log Started 2015-04-28T04:44:04Z ***********************
*********************** Log Started 2015-04-28T04:44:04Z ***********************
10:34:03:WU01:FS00:Connecting to 171.67.108.200:8080
10:34:05:WU01:FS00:Assigned to work server 171.64.65.124
10:34:05:WU01:FS00:Requesting new work unit for slot 00: RUNNING cpu:3 from 171.64.65.124
10:34:05:WU01:FS00:Connecting to 171.64.65.124:8080
10:34:06:WU01:FS00:Downloading 902.50KiB
10:34:07:WU01:FS00:Download complete
10:34:07:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:9015 run:488 clone:1 gen:197 core:0xa4 unit:0x000000f1664f2de453e55e41f721b973
10:39:17:WU01:FS00:Starting
10:39:17:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/John/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 704 -lifeline 5552 -checkpoint 15 -np 3
10:39:17:WU01:FS00:Started FahCore on PID 5344
10:39:17:WU01:FS00:Core PID:8108
10:39:17:WU01:FS00:FahCore 0xa4 started
10:39:18:WU01:FS00:0xa4:
10:39:18:WU01:FS00:0xa4:*------------------------------*
10:39:18:WU01:FS00:0xa4:Folding@Home Gromacs GB Core
10:39:18:WU01:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
10:39:18:WU01:FS00:0xa4:
10:39:18:WU01:FS00:0xa4:Preparing to commence simulation
10:39:18:WU01:FS00:0xa4:- Looking at optimizations...
10:39:18:WU01:FS00:0xa4:- Created dyn
10:39:18:WU01:FS00:0xa4:- Files status OK
10:39:18:WU01:FS00:0xa4:- Expanded 923647 -> 1526808 (decompressed 165.3 percent)
10:39:18:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=923647 data_size=1526808, decompressed_data_size=1526808 diff=0
10:39:18:WU01:FS00:0xa4:- Digital signature verified
10:39:18:WU01:FS00:0xa4:
10:39:18:WU01:FS00:0xa4:Project: 9015 (Run 488, Clone 1, Gen 197)
10:39:18:WU01:FS00:0xa4:
10:39:18:WU01:FS00:0xa4:Assembly optimizations on if available.
10:39:18:WU01:FS00:0xa4:Entering M.D.
10:39:24:WU01:FS00:0xa4:Mapping NT from 3 to 3 
10:39:24:WU01:FS00:0xa4:Completed 0 out of 250000 steps  (0%)
10:43:39:WU01:FS00:0xa4:Completed 2500 out of 250000 steps  (1%)
******************************* Date: 2015-04-28 *******************************
10:47:55:WU01:FS00:0xa4:Completed 5000 out of 250000 steps  (2%)
10:52:05:WU01:FS00:0xa4:Completed 7500 out of 250000 steps  (3%)
10:56:21:WU01:FS00:0xa4:Completed 10000 out of 250000 steps  (4%)
11:00:37:WU01:FS00:0xa4:Completed 12500 out of 250000 steps  (5%)
11:05:25:WU01:FS00:0xa4:Completed 15000 out of 250000 steps  (6%)
11:09:44:WU01:FS00:0xa4:Completed 17500 out of 250000 steps  (7%)
11:13:57:WU01:FS00:0xa4:Completed 20000 out of 250000 steps  (8%)
11:18:14:WU01:FS00:0xa4:Completed 22500 out of 250000 steps  (9%)
11:22:26:WU01:FS00:0xa4:Completed 25000 out of 250000 steps  (10%)
11:26:43:WU01:FS00:0xa4:Completed 27500 out of 250000 steps  (11%)
11:31:00:WU01:FS00:0xa4:Completed 30000 out of 250000 steps  (12%)
11:35:12:WU01:FS00:0xa4:Completed 32500 out of 250000 steps  (13%)
11:39:25:WU01:FS00:0xa4:Completed 35000 out of 250000 steps  (14%)
11:43:38:WU01:FS00:0xa4:Completed 37500 out of 250000 steps  (15%)
11:47:57:WU01:FS00:0xa4:Completed 40000 out of 250000 steps  (16%)
11:52:09:WU01:FS00:0xa4:Completed 42500 out of 250000 steps  (17%)
11:56:24:WU01:FS00:0xa4:Completed 45000 out of 250000 steps  (18%)
12:00:39:WU01:FS00:0xa4:Completed 47500 out of 250000 steps  (19%)
12:04:55:WU01:FS00:0xa4:Completed 50000 out of 250000 steps  (20%)
12:09:08:WU01:FS00:0xa4:Completed 52500 out of 250000 steps  (21%)
12:13:22:WU01:FS00:0xa4:Completed 55000 out of 250000 steps  (22%)
12:17:37:WU01:FS00:0xa4:Completed 57500 out of 250000 steps  (23%)
12:21:54:WU01:FS00:0xa4:Completed 60000 out of 250000 steps  (24%)
12:26:08:WU01:FS00:0xa4:Completed 62500 out of 250000 steps  (25%)
12:30:20:WU01:FS00:0xa4:Completed 65000 out of 250000 steps  (26%)
12:34:32:WU01:FS00:0xa4:Completed 67500 out of 250000 steps  (27%)
12:38:43:WU01:FS00:0xa4:Completed 70000 out of 250000 steps  (28%)
12:42:59:WU01:FS00:0xa4:Completed 72500 out of 250000 steps  (29%)
12:47:11:WU01:FS00:0xa4:Completed 75000 out of 250000 steps  (30%)
12:51:22:WU01:FS00:0xa4:Completed 77500 out of 250000 steps  (31%)
12:55:38:WU01:FS00:0xa4:Completed 80000 out of 250000 steps  (32%)
12:59:49:WU01:FS00:0xa4:Completed 82500 out of 250000 steps  (33%)
13:04:02:WU01:FS00:0xa4:Completed 85000 out of 250000 steps  (34%)
13:08:23:WU01:FS00:0xa4:Completed 87500 out of 250000 steps  (35%)
13:12:37:WU01:FS00:0xa4:Completed 90000 out of 250000 steps  (36%)
13:16:52:WU01:FS00:0xa4:Completed 92500 out of 250000 steps  (37%)
13:21:04:WU01:FS00:0xa4:Completed 95000 out of 250000 steps  (38%)
13:25:14:WU01:FS00:0xa4:Completed 97500 out of 250000 steps  (39%)
13:29:26:WU01:FS00:0xa4:Completed 100000 out of 250000 steps  (40%)
13:33:38:WU01:FS00:0xa4:Completed 102500 out of 250000 steps  (41%)
13:37:57:WU01:FS00:0xa4:Completed 105000 out of 250000 steps  (42%)
13:42:09:WU01:FS00:0xa4:Completed 107500 out of 250000 steps  (43%)
13:46:26:WU01:FS00:0xa4:Completed 110000 out of 250000 steps  (44%)
13:50:45:WU01:FS00:0xa4:Completed 112500 out of 250000 steps  (45%)
13:55:01:WU01:FS00:0xa4:Completed 115000 out of 250000 steps  (46%)
13:59:15:WU01:FS00:0xa4:Completed 117500 out of 250000 steps  (47%)
14:03:34:WU01:FS00:0xa4:Completed 120000 out of 250000 steps  (48%)
14:07:48:WU01:FS00:0xa4:Completed 122500 out of 250000 steps  (49%)
14:12:04:WU01:FS00:0xa4:Completed 125000 out of 250000 steps  (50%)
14:16:18:WU01:FS00:0xa4:Completed 127500 out of 250000 steps  (51%)
14:20:29:WU01:FS00:0xa4:Completed 130000 out of 250000 steps  (52%)
14:24:49:WU01:FS00:0xa4:Completed 132500 out of 250000 steps  (53%)
14:29:02:WU01:FS00:0xa4:Completed 135000 out of 250000 steps  (54%)
14:33:14:WU01:FS00:0xa4:Completed 137500 out of 250000 steps  (55%)
14:37:35:WU01:FS00:0xa4:Completed 140000 out of 250000 steps  (56%)
14:41:47:WU01:FS00:0xa4:Completed 142500 out of 250000 steps  (57%)
14:46:02:WU01:FS00:0xa4:Completed 145000 out of 250000 steps  (58%)
14:50:16:WU01:FS00:0xa4:Completed 147500 out of 250000 steps  (59%)
14:54:28:WU01:FS00:0xa4:Completed 150000 out of 250000 steps  (60%)
14:58:41:WU01:FS00:0xa4:Completed 152500 out of 250000 steps  (61%)
15:03:00:WU01:FS00:0xa4:Completed 155000 out of 250000 steps  (62%)
15:07:13:WU01:FS00:0xa4:Completed 157500 out of 250000 steps  (63%)
15:11:28:WU01:FS00:0xa4:Completed 160000 out of 250000 steps  (64%)
15:15:45:WU01:FS00:0xa4:Completed 162500 out of 250000 steps  (65%)
15:19:57:WU01:FS00:0xa4:Completed 165000 out of 250000 steps  (66%)
15:24:18:WU01:FS00:0xa4:Completed 167500 out of 250000 steps  (67%)
15:28:28:WU01:FS00:0xa4:Completed 170000 out of 250000 steps  (68%)
15:32:39:WU01:FS00:0xa4:Completed 172500 out of 250000 steps  (69%)
15:37:01:WU01:FS00:0xa4:Completed 175000 out of 250000 steps  (70%)
15:41:15:WU01:FS00:0xa4:Completed 177500 out of 250000 steps  (71%)
15:45:31:WU01:FS00:0xa4:Completed 180000 out of 250000 steps  (72%)
15:49:51:WU01:FS00:0xa4:Completed 182500 out of 250000 steps  (73%)
15:54:02:WU01:FS00:0xa4:Completed 185000 out of 250000 steps  (74%)
15:58:14:WU01:FS00:0xa4:Completed 187500 out of 250000 steps  (75%)
16:02:33:WU01:FS00:0xa4:Completed 190000 out of 250000 steps  (76%)
16:06:47:WU01:FS00:0xa4:Completed 192500 out of 250000 steps  (77%)
16:10:58:WU01:FS00:0xa4:Completed 195000 out of 250000 steps  (78%)
16:15:18:WU01:FS00:0xa4:Completed 197500 out of 250000 steps  (79%)
16:19:30:WU01:FS00:0xa4:Completed 200000 out of 250000 steps  (80%)
16:23:42:WU01:FS00:0xa4:Completed 202500 out of 250000 steps  (81%)
16:27:54:WU01:FS00:0xa4:Completed 205000 out of 250000 steps  (82%)
16:32:16:WU01:FS00:0xa4:Completed 207500 out of 250000 steps  (83%)
16:36:33:WU01:FS00:0xa4:Completed 210000 out of 250000 steps  (84%)
16:40:46:WU01:FS00:0xa4:Completed 212500 out of 250000 steps  (85%)
16:45:15:WU01:FS00:0xa4:Completed 215000 out of 250000 steps  (86%)
******************************* Date: 2015-04-28 *******************************
16:49:29:WU01:FS00:0xa4:Completed 217500 out of 250000 steps  (87%)
16:53:42:WU01:FS00:0xa4:Completed 220000 out of 250000 steps  (88%)
16:57:52:WU01:FS00:0xa4:Completed 222500 out of 250000 steps  (89%)
17:02:01:WU01:FS00:0xa4:Completed 225000 out of 250000 steps  (90%)
17:06:57:WU01:FS00:0xa4:Completed 227500 out of 250000 steps  (91%)
17:11:23:WU01:FS00:0xa4:Completed 230000 out of 250000 steps  (92%)
17:15:35:WU01:FS00:0xa4:Completed 232500 out of 250000 steps  (93%)
17:19:47:WU01:FS00:0xa4:Completed 235000 out of 250000 steps  (94%)
17:23:57:WU01:FS00:0xa4:Completed 237500 out of 250000 steps  (95%)
17:28:32:WU01:FS00:0xa4:Completed 240000 out of 250000 steps  (96%)
17:33:00:WU01:FS00:0xa4:Completed 242500 out of 250000 steps  (97%)
17:37:40:WU01:FS00:0xa4:Completed 245000 out of 250000 steps  (98%)
17:42:17:WU01:FS00:0xa4:Completed 247500 out of 250000 steps  (99%)
17:46:49:WU01:FS00:0xa4:Completed 250000 out of 250000 steps  (100%)
17:46:49:WU01:FS00:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
17:46:59:WU01:FS00:0xa4:
17:46:59:WU01:FS00:0xa4:Finished Work Unit:
17:46:59:WU01:FS00:0xa4:- Reading up to 916296 from "01/wudata_01.trr": Read 916296
17:46:59:WU01:FS00:0xa4:trr file hash check passed.
17:46:59:WU01:FS00:0xa4:- Reading up to 839716 from "01/wudata_01.xtc": Read 839716
17:46:59:WU01:FS00:0xa4:xtc file hash check passed.
17:46:59:WU01:FS00:0xa4:edr file hash check passed.
17:46:59:WU01:FS00:0xa4:logfile size: 24190
17:46:59:WU01:FS00:0xa4:Leaving Run
17:47:01:WU01:FS00:0xa4:- Writing 1782690 bytes of core data to disk...
17:47:01:WU01:FS00:0xa4:Done: 1782178 -> 1722491 (compressed to 96.6 percent)
17:47:01:WU01:FS00:0xa4:  ... Done.
17:47:02:WU01:FS00:0xa4:- Shutting down core
17:47:02:WU01:FS00:0xa4:
17:47:02:WU01:FS00:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
17:47:03:WU01:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
17:47:03:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:9015 run:488 clone:1 gen:197 core:0xa4 unit:0x000000f1664f2de453e55e41f721b973
17:47:03:WU01:FS00:Uploading 1.64MiB to 171.64.65.124
17:47:03:WU01:FS00:Connecting to 171.64.65.124:8080
17:56:51:WARNING:WU01:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
17:56:51:WU01:FS00:Trying to send results to collection server
17:56:51:WU01:FS00:Uploading 1.64MiB to 171.65.103.160
17:56:51:WU01:FS00:Connecting to 171.65.103.160:8080
17:58:07:ERROR:WU01:FS00:Exception: 10002: Received short response, expected 512 bytes, got 0
17:58:08:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:9015 run:488 clone:1 gen:197 core:0xa4 unit:0x000000f1664f2de453e55e41f721b973
17:58:08:WU01:FS00:Uploading 1.64MiB to 171.64.65.124
17:58:08:WU01:FS00:Connecting to 171.64.65.124:8080
17:58:09:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
17:58:09:WU01:FS00:Connecting to 171.64.65.124:80
17:58:10:WARNING:WU01:FS00:Exception: Failed to send results to work server: Failed to connect to 171.64.65.124:80: No connection could be made because the target machine actively refused it.
17:58:10:WU01:FS00:Trying to send results to collection server
17:58:10:WU01:FS00:Uploading 1.64MiB to 171.65.103.160
17:58:10:WU01:FS00:Connecting to 171.65.103.160:8080
17:58:15:WU01:FS00:Upload complete
17:58:15:WU01:FS00:Server responded WORK_ACK (400)
17:58:15:WU01:FS00:Final credit estimate, 1030.00 points
17:58:15:WU01:FS00:Cleaning up
******************************* Date: 2015-04-28 *******************************
01:48:32:WU01:FS00:Connecting to 171.67.108.200:8080
01:48:32:WU01:FS00:Assigned to work server 171.64.65.124
01:48:32:WU01:FS00:Requesting new work unit for slot 00: RUNNING cpu:3 from 171.64.65.124
01:48:32:WU01:FS00:Connecting to 171.64.65.124:8080
01:48:33:WU01:FS00:Downloading 807.58KiB
01:48:33:WU01:FS00:Download complete
01:48:34:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:9016 run:405 clone:3 gen:155 core:0xa4 unit:0x000000b4664f2de45491d98025b640f8
01:53:32:WU01:FS00:Starting
01:53:32:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/John/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 704 -lifeline 5552 -checkpoint 15 -np 3
01:53:33:WU01:FS00:Started FahCore on PID 3680
01:53:33:WU01:FS00:Core PID:1580
01:53:33:WU01:FS00:FahCore 0xa4 started
01:53:33:WU01:FS00:0xa4:
01:53:33:WU01:FS00:0xa4:*------------------------------*
01:53:33:WU01:FS00:0xa4:Folding@Home Gromacs GB Core
01:53:33:WU01:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
01:53:33:WU01:FS00:0xa4:
01:53:33:WU01:FS00:0xa4:Preparing to commence simulation
01:53:33:WU01:FS00:0xa4:- Looking at optimizations...
01:53:33:WU01:FS00:0xa4:- Created dyn
01:53:33:WU01:FS00:0xa4:- Files status OK
01:53:33:WU01:FS00:0xa4:- Expanded 826451 -> 1397548 (decompressed 169.1 percent)
01:53:33:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=826451 data_size=1397548, decompressed_data_size=1397548 diff=0
01:53:33:WU01:FS00:0xa4:- Digital signature verified
01:53:33:WU01:FS00:0xa4:
01:53:33:WU01:FS00:0xa4:Project: 9016 (Run 405, Clone 3, Gen 155)
01:53:33:WU01:FS00:0xa4:
01:53:33:WU01:FS00:0xa4:Assembly optimizations on if available.
01:53:33:WU01:FS00:0xa4:Entering M.D.
01:53:39:WU01:FS00:0xa4:Mapping NT from 3 to 3 
01:53:40:WU01:FS00:0xa4:Completed 0 out of 250000 steps  (0%)
01:57:20:WU01:FS00:0xa4:Completed 2500 out of 250000 steps  (1%)
02:00:59:WU01:FS00:0xa4:Completed 5000 out of 250000 steps  (2%)
02:04:36:WU01:FS00:0xa4:Completed 7500 out of 250000 steps  (3%)
02:08:15:WU01:FS00:0xa4:Completed 10000 out of 250000 steps  (4%)
02:11:54:WU01:FS00:0xa4:Completed 12500 out of 250000 steps  (5%)
02:15:33:WU01:FS00:0xa4:Completed 15000 out of 250000 steps  (6%)
02:19:12:WU01:FS00:0xa4:Completed 17500 out of 250000 steps  (7%)
02:22:50:WU01:FS00:0xa4:Completed 20000 out of 250000 steps  (8%)
02:26:30:WU01:FS00:0xa4:Completed 22500 out of 250000 steps  (9%)
02:30:08:WU01:FS00:0xa4:Completed 25000 out of 250000 steps  (10%)
02:33:47:WU01:FS00:0xa4:Completed 27500 out of 250000 steps  (11%)
02:37:27:WU01:FS00:0xa4:Completed 30000 out of 250000 steps  (12%)
02:41:07:WU01:FS00:0xa4:Completed 32500 out of 250000 steps  (13%)
02:44:47:WU01:FS00:0xa4:Completed 35000 out of 250000 steps  (14%)
02:48:28:WU01:FS00:0xa4:Completed 37500 out of 250000 steps  (15%)
02:52:08:WU01:FS00:0xa4:Completed 40000 out of 250000 steps  (16%)
02:55:49:WU01:FS00:0xa4:Completed 42500 out of 250000 steps  (17%)
02:59:28:WU01:FS00:0xa4:Completed 45000 out of 250000 steps  (18%)
03:03:07:WU01:FS00:0xa4:Completed 47500 out of 250000 steps  (19%)
03:06:47:WU01:FS00:0xa4:Completed 50000 out of 250000 steps  (20%)
03:10:27:WU01:FS00:0xa4:Completed 52500 out of 250000 steps  (21%)
03:14:06:WU01:FS00:0xa4:Completed 55000 out of 250000 steps  (22%)
03:17:46:WU01:FS00:0xa4:Completed 57500 out of 250000 steps  (23%)
03:21:26:WU01:FS00:0xa4:Completed 60000 out of 250000 steps  (24%)
03:25:05:WU01:FS00:0xa4:Completed 62500 out of 250000 steps  (25%)
03:28:46:WU01:FS00:0xa4:Completed 65000 out of 250000 steps  (26%)
03:32:26:WU01:FS00:0xa4:Completed 67500 out of 250000 steps  (27%)
03:36:05:WU01:FS00:0xa4:Completed 70000 out of 250000 steps  (28%)
03:39:46:WU01:FS00:0xa4:Completed 72500 out of 250000 steps  (29%)
03:43:26:WU01:FS00:0xa4:Completed 75000 out of 250000 steps  (30%)
03:47:06:WU01:FS00:0xa4:Completed 77500 out of 250000 steps  (31%)
03:50:44:WU01:FS00:0xa4:Completed 80000 out of 250000 steps  (32%)
03:54:24:WU01:FS00:0xa4:Completed 82500 out of 250000 steps  (33%)
03:58:03:WU01:FS00:0xa4:Completed 85000 out of 250000 steps  (34%)
04:01:42:WU01:FS00:0xa4:Completed 87500 out of 250000 steps  (35%)
04:05:20:WU01:FS00:0xa4:Completed 90000 out of 250000 steps  (36%)
04:08:59:WU01:FS00:0xa4:Completed 92500 out of 250000 steps  (37%)
04:12:38:WU01:FS00:0xa4:Completed 95000 out of 250000 steps  (38%)
04:16:15:WU01:FS00:0xa4:Completed 97500 out of 250000 steps  (39%)
04:19:53:WU01:FS00:0xa4:Completed 100000 out of 250000 steps  (40%)
04:23:31:WU01:FS00:0xa4:Completed 102500 out of 250000 steps  (41%)
04:27:10:WU01:FS00:0xa4:Completed 105000 out of 250000 steps  (42%)
04:30:49:WU01:FS00:0xa4:Completed 107500 out of 250000 steps  (43%)
04:34:28:WU01:FS00:0xa4:Completed 110000 out of 250000 steps  (44%)
04:38:06:WU01:FS00:0xa4:Completed 112500 out of 250000 steps  (45%)
04:41:45:WU01:FS00:0xa4:Completed 115000 out of 250000 steps  (46%)
04:45:25:WU01:FS00:0xa4:Completed 117500 out of 250000 steps  (47%)
04:49:06:WU01:FS00:0xa4:Completed 120000 out of 250000 steps  (48%)
******************************* Date: 2015-04-29 *******************************
04:52:46:WU01:FS00:0xa4:Completed 122500 out of 250000 steps  (49%)
04:56:37:WU01:FS00:0xa4:Completed 125000 out of 250000 steps  (50%)
05:00:32:WU01:FS00:0xa4:Completed 127500 out of 250000 steps  (51%)
05:04:58:WU01:FS00:0xa4:Completed 130000 out of 250000 steps  (52%)
05:08:56:WU01:FS00:0xa4:Completed 132500 out of 250000 steps  (53%)
05:12:48:WU01:FS00:0xa4:Completed 135000 out of 250000 steps  (54%)
05:16:41:WU01:FS00:0xa4:Completed 137500 out of 250000 steps  (55%)
05:20:34:WU01:FS00:0xa4:Completed 140000 out of 250000 steps  (56%)
05:24:27:WU01:FS00:0xa4:Completed 142500 out of 250000 steps  (57%)
05:28:25:WU01:FS00:0xa4:Completed 145000 out of 250000 steps  (58%)
05:32:18:WU01:FS00:0xa4:Completed 147500 out of 250000 steps  (59%)
05:36:15:WU01:FS00:0xa4:Completed 150000 out of 250000 steps  (60%)
05:40:11:WU01:FS00:0xa4:Completed 152500 out of 250000 steps  (61%)
05:44:05:WU01:FS00:0xa4:Completed 155000 out of 250000 steps  (62%)
05:48:01:WU01:FS00:0xa4:Completed 157500 out of 250000 steps  (63%)
05:51:56:WU01:FS00:0xa4:Completed 160000 out of 250000 steps  (64%)
05:55:52:WU01:FS00:0xa4:Completed 162500 out of 250000 steps  (65%)
05:59:49:WU01:FS00:0xa4:Completed 165000 out of 250000 steps  (66%)
06:03:44:WU01:FS00:0xa4:Completed 167500 out of 250000 steps  (67%)
06:07:40:WU01:FS00:0xa4:Completed 170000 out of 250000 steps  (68%)
06:11:36:WU01:FS00:0xa4:Completed 172500 out of 250000 steps  (69%)
06:15:31:WU01:FS00:0xa4:Completed 175000 out of 250000 steps  (70%)
06:19:25:WU01:FS00:0xa4:Completed 177500 out of 250000 steps  (71%)
06:23:21:WU01:FS00:0xa4:Completed 180000 out of 250000 steps  (72%)
06:27:16:WU01:FS00:0xa4:Completed 182500 out of 250000 steps  (73%)
06:31:11:WU01:FS00:0xa4:Completed 185000 out of 250000 steps  (74%)
06:35:07:WU01:FS00:0xa4:Completed 187500 out of 250000 steps  (75%)
06:39:04:WU01:FS00:0xa4:Completed 190000 out of 250000 steps  (76%)
06:43:00:WU01:FS00:0xa4:Completed 192500 out of 250000 steps  (77%)
06:46:57:WU01:FS00:0xa4:Completed 195000 out of 250000 steps  (78%)
06:50:53:WU01:FS00:0xa4:Completed 197500 out of 250000 steps  (79%)
06:54:50:WU01:FS00:0xa4:Completed 200000 out of 250000 steps  (80%)
06:58:48:WU01:FS00:0xa4:Completed 202500 out of 250000 steps  (81%)
07:02:48:WU01:FS00:0xa4:Completed 205000 out of 250000 steps  (82%)
07:06:48:WU01:FS00:0xa4:Completed 207500 out of 250000 steps  (83%)
07:10:46:WU01:FS00:0xa4:Completed 210000 out of 250000 steps  (84%)
07:14:41:WU01:FS00:0xa4:Completed 212500 out of 250000 steps  (85%)
07:18:36:WU01:FS00:0xa4:Completed 215000 out of 250000 steps  (86%)
07:22:30:WU01:FS00:0xa4:Completed 217500 out of 250000 steps  (87%)
07:26:23:WU01:FS00:0xa4:Completed 220000 out of 250000 steps  (88%)
07:30:54:WU01:FS00:0xa4:Completed 222500 out of 250000 steps  (89%)
07:36:22:WU01:FS00:0xa4:Completed 225000 out of 250000 steps  (90%)
07:41:54:WU01:FS00:0xa4:Completed 227500 out of 250000 steps  (91%)
07:45:48:WU01:FS00:0xa4:Completed 230000 out of 250000 steps  (92%)
07:50:00:WU01:FS00:0xa4:Completed 232500 out of 250000 steps  (93%)
07:54:24:WU01:FS00:0xa4:Completed 235000 out of 250000 steps  (94%)
07:58:30:WU01:FS00:0xa4:Completed 237500 out of 250000 steps  (95%)
08:02:27:WU01:FS00:0xa4:Completed 240000 out of 250000 steps  (96%)
08:06:26:WU01:FS00:0xa4:Completed 242500 out of 250000 steps  (97%)
08:10:34:WU01:FS00:0xa4:Completed 245000 out of 250000 steps  (98%)
08:14:38:WU01:FS00:0xa4:Completed 247500 out of 250000 steps  (99%)
08:19:27:WU01:FS00:0xa4:Completed 250000 out of 250000 steps  (100%)
08:19:28:WU01:FS00:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
08:19:38:WU01:FS00:0xa4:
08:19:38:WU01:FS00:0xa4:Finished Work Unit:
08:19:38:WU01:FS00:0xa4:- Reading up to 811488 from "01/wudata_01.trr": Read 811488
08:19:38:WU01:FS00:0xa4:trr file hash check passed.
08:19:38:WU01:FS00:0xa4:- Reading up to 746380 from "01/wudata_01.xtc": Read 746380
08:19:38:WU01:FS00:0xa4:xtc file hash check passed.
08:19:38:WU01:FS00:0xa4:edr file hash check passed.
08:19:38:WU01:FS00:0xa4:logfile size: 24002
08:19:38:WU01:FS00:0xa4:Leaving Run
08:19:43:WU01:FS00:0xa4:- Writing 1584358 bytes of core data to disk...
08:19:44:WU01:FS00:0xa4:Done: 1583846 -> 1538190 (compressed to 97.1 percent)
08:19:44:WU01:FS00:0xa4:  ... Done.
08:19:45:WU01:FS00:0xa4:- Shutting down core
08:19:45:WU01:FS00:0xa4:
08:19:45:WU01:FS00:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
08:19:46:WU01:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
08:19:46:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:9016 run:405 clone:3 gen:155 core:0xa4 unit:0x000000b4664f2de45491d98025b640f8
08:19:46:WU01:FS00:Uploading 1.47MiB to 171.64.65.124
08:19:46:WU01:FS00:Connecting to 171.64.65.124:8080
******************************* Date: 2015-04-29 *******************************
******************************* Date: 2015-04-29 *******************************
******************************* Date: 2015-04-29 *******************************
******************************* Date: 2015-04-30 *******************************
******************************* Date: 2015-04-30 *******************************
******************************* Date: 2015-04-30 *******************************
******************************* Date: 2015-04-29 *******************************
******************************* Date: 2015-04-30 *******************************
******************************* Date: 2015-04-30 *******************************
******************************* Date: 2015-04-30 *******************************
******************************* Date: 2015-04-30 *******************************
******************************* Date: 2015-05-01 *******************************
******************************* Date: 2015-05-01 *******************************
******************************* Date: 2015-05-01 *******************************
******************************* Date: 2015-05-01 *******************************
******************************* Date: 2015-05-02 *******************************
mod edit: Added CODE tags.
Donk_Enterprises
Posts: 4
Joined: Sat May 02, 2015 5:13 am

Re: WU 9016 stuck at "Sending"

Post by Donk_Enterprises »

And here's the slot 00 log that will show successful uploading after the failure.

Code: Select all

*********************** Log Started 2015-04-28T04:44:04Z ***********************
04:46:19:FS00:Unpaused
04:46:20:WU02:FS00:Starting
04:46:20:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/John/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 02 -suffix 01 -version 704 -lifeline 5552 -checkpoint 15 -np 3
04:46:20:WU02:FS00:Started FahCore on PID 7472
04:46:20:WU02:FS00:Core PID:7388
04:46:20:WU02:FS00:FahCore 0xa4 started
04:46:21:WU02:FS00:0xa4:
04:46:21:WU02:FS00:0xa4:*------------------------------*
04:46:21:WU02:FS00:0xa4:Folding@Home Gromacs GB Core
04:46:21:WU02:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
04:46:21:WU02:FS00:0xa4:
04:46:21:WU02:FS00:0xa4:Preparing to commence simulation
04:46:21:WU02:FS00:0xa4:- Looking at optimizations...
04:46:21:WU02:FS00:0xa4:- Files status OK
04:46:21:WU02:FS00:0xa4:- Expanded 118788 -> 269764 (decompressed 227.0 percent)
04:46:21:WU02:FS00:0xa4:Called DecompressByteArray: compressed_data_size=118788 data_size=269764, decompressed_data_size=269764 diff=0
04:46:21:WU02:FS00:0xa4:- Digital signature verified
04:46:21:WU02:FS00:0xa4:
04:46:21:WU02:FS00:0xa4:Project: 6399 (Run 101, Clone 14, Gen 66)
04:46:21:WU02:FS00:0xa4:
04:46:21:WU02:FS00:0xa4:Assembly optimizations on if available.
04:46:21:WU02:FS00:0xa4:Entering M.D.
04:46:26:WU02:FS00:0xa4:Using Gromacs checkpoints
04:46:26:WU02:FS00:0xa4:Mapping NT from 3 to 3 
04:46:27:WU02:FS00:0xa4:Resuming from checkpoint
04:46:27:WU02:FS00:0xa4:Verified 02/wudata_01.log
04:46:27:WU02:FS00:0xa4:Verified 02/wudata_01.trr
04:46:27:WU02:FS00:0xa4:Verified 02/wudata_01.xtc
04:46:27:WU02:FS00:0xa4:Verified 02/wudata_01.edr
04:46:27:WU02:FS00:0xa4:Completed 2012280 out of 5000000 steps  (40%)
04:50:37:WU02:FS00:0xa4:Completed 2050000 out of 5000000 steps  (41%)
...
09:08:49:WU02:FS00:0xa4:Completed 4200000 out of 5000000 steps  (84%)
09:14:03:WU02:FS00:0xa4:Completed 4250000 out of 5000000 steps  (85%)
*********************** Log Started 2015-04-28T04:44:04Z ***********************
04:46:19:FS00:Unpaused
04:46:20:WU02:FS00:Starting
04:46:20:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/John/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 02 -suffix 01 -version 704 -lifeline 5552 -checkpoint 15 -np 3
04:46:20:WU02:FS00:Started FahCore on PID 7472
04:46:20:WU02:FS00:Core PID:7388
04:46:20:WU02:FS00:FahCore 0xa4 started
04:46:21:WU02:FS00:0xa4:
04:46:21:WU02:FS00:0xa4:*------------------------------*
04:46:21:WU02:FS00:0xa4:Folding@Home Gromacs GB Core
04:46:21:WU02:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
04:46:21:WU02:FS00:0xa4:
04:46:21:WU02:FS00:0xa4:Preparing to commence simulation
04:46:21:WU02:FS00:0xa4:- Looking at optimizations...
04:46:21:WU02:FS00:0xa4:- Files status OK
04:46:21:WU02:FS00:0xa4:- Expanded 118788 -> 269764 (decompressed 227.0 percent)
04:46:21:WU02:FS00:0xa4:Called DecompressByteArray: compressed_data_size=118788 data_size=269764, decompressed_data_size=269764 diff=0
04:46:21:WU02:FS00:0xa4:- Digital signature verified
04:46:21:WU02:FS00:0xa4:
04:46:21:WU02:FS00:0xa4:Project: 6399 (Run 101, Clone 14, Gen 66)
04:46:21:WU02:FS00:0xa4:
04:46:21:WU02:FS00:0xa4:Assembly optimizations on if available.
04:46:21:WU02:FS00:0xa4:Entering M.D.
04:46:26:WU02:FS00:0xa4:Using Gromacs checkpoints
04:46:26:WU02:FS00:0xa4:Mapping NT from 3 to 3 
04:46:27:WU02:FS00:0xa4:Resuming from checkpoint
04:46:27:WU02:FS00:0xa4:Verified 02/wudata_01.log
04:46:27:WU02:FS00:0xa4:Verified 02/wudata_01.trr
04:46:27:WU02:FS00:0xa4:Verified 02/wudata_01.xtc
04:46:27:WU02:FS00:0xa4:Verified 02/wudata_01.edr
04:46:27:WU02:FS00:0xa4:Completed 2012280 out of 5000000 steps  (40%)
...
10:34:01:WU02:FS00:0xa4:Completed 4950000 out of 5000000 steps  (99%)
10:34:03:WU01:FS00:Connecting to 171.67.108.200:8080
10:34:05:WU01:FS00:Assigned to work server 171.64.65.124
10:34:05:WU01:FS00:Requesting new work unit for slot 00: RUNNING cpu:3 from 171.64.65.124
10:34:05:WU01:FS00:Connecting to 171.64.65.124:8080
10:34:06:WU01:FS00:Downloading 902.50KiB
10:34:07:WU01:FS00:Download complete
10:34:07:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:9015 run:488 clone:1 gen:197 core:0xa4 unit:0x000000f1664f2de453e55e41f721b973
10:39:00:WU02:FS00:0xa4:Completed 5000000 out of 5000000 steps  (100%)
10:39:01:WU02:FS00:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
10:39:11:WU02:FS00:0xa4:
10:39:11:WU02:FS00:0xa4:Finished Work Unit:
10:39:11:WU02:FS00:0xa4:- Reading up to 1255848 from "02/wudata_01.trr": Read 1255848
10:39:11:WU02:FS00:0xa4:trr file hash check passed.
10:39:11:WU02:FS00:0xa4:- Reading up to 110436 from "02/wudata_01.xtc": Read 110436
10:39:11:WU02:FS00:0xa4:xtc file hash check passed.
10:39:11:WU02:FS00:0xa4:edr file hash check passed.
10:39:11:WU02:FS00:0xa4:logfile size: 89556
10:39:11:WU02:FS00:0xa4:Leaving Run
10:39:15:WU02:FS00:0xa4:- Writing 1527740 bytes of core data to disk...
10:39:16:WU02:FS00:0xa4:Done: 1527228 -> 1291915 (compressed to 84.5 percent)
10:39:16:WU02:FS00:0xa4:  ... Done.
10:39:16:WU02:FS00:0xa4:- Shutting down core
10:39:16:WU02:FS00:0xa4:
10:39:16:WU02:FS00:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
10:39:17:WU02:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
10:39:17:WU02:FS00:Sending unit results: id:02 state:SEND error:NO_ERROR project:6399 run:101 clone:14 gen:66 core:0xa4 unit:0x0000004a0002894b5462ccabcc55882b
10:39:17:WU02:FS00:Uploading 1.23MiB to 155.247.166.219
10:39:17:WU01:FS00:Starting
10:39:17:WU02:FS00:Connecting to 155.247.166.219:8080
10:39:17:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/John/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 704 -lifeline 5552 -checkpoint 15 -np 3
10:39:17:WU01:FS00:Started FahCore on PID 5344
10:39:17:WU01:FS00:Core PID:8108
10:39:17:WU01:FS00:FahCore 0xa4 started
10:39:18:WU01:FS00:0xa4:
10:39:18:WU01:FS00:0xa4:*------------------------------*
10:39:18:WU01:FS00:0xa4:Folding@Home Gromacs GB Core
10:39:18:WU01:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
10:39:18:WU01:FS00:0xa4:
10:39:18:WU01:FS00:0xa4:Preparing to commence simulation
10:39:18:WU01:FS00:0xa4:- Looking at optimizations...
10:39:18:WU01:FS00:0xa4:- Created dyn
10:39:18:WU01:FS00:0xa4:- Files status OK
10:39:18:WU01:FS00:0xa4:- Expanded 923647 -> 1526808 (decompressed 165.3 percent)
10:39:18:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=923647 data_size=1526808, decompressed_data_size=1526808 diff=0
10:39:18:WU01:FS00:0xa4:- Digital signature verified
10:39:18:WU01:FS00:0xa4:
10:39:18:WU01:FS00:0xa4:Project: 9015 (Run 488, Clone 1, Gen 197)
10:39:18:WU01:FS00:0xa4:
10:39:18:WU01:FS00:0xa4:Assembly optimizations on if available.
10:39:18:WU01:FS00:0xa4:Entering M.D.
10:39:23:WU02:FS00:Upload 55.78%
10:39:24:WU01:FS00:0xa4:Mapping NT from 3 to 3 
10:39:24:WU01:FS00:0xa4:Completed 0 out of 250000 steps  (0%)
10:39:27:WU02:FS00:Upload complete
10:39:28:WU02:FS00:Server responded WORK_ACK (400)
10:39:28:WU02:FS00:Final credit estimate, 719.00 points
10:39:28:WU02:FS00:Cleaning up
10:43:39:WU01:FS00:0xa4:Completed 2500 out of 250000 steps  (1%)
******************************* Date: 2015-04-28 *******************************
10:47:55:WU01:FS00:0xa4:Completed 5000 out of 250000 steps  (2%)
...
16:45:15:WU01:FS00:0xa4:Completed 215000 out of 250000 steps  (86%)
******************************* Date: 2015-04-28 *******************************
16:49:29:WU01:FS00:0xa4:Completed 217500 out of 250000 steps  (87%)
...
17:42:17:WU01:FS00:0xa4:Completed 247500 out of 250000 steps  (99%)
17:42:19:WU00:FS00:Connecting to 171.67.108.200:8080
17:42:19:WU00:FS00:Assigned to work server 155.247.166.219
17:42:19:WU00:FS00:Requesting new work unit for slot 00: RUNNING cpu:3 from 155.247.166.219
17:42:19:WU00:FS00:Connecting to 155.247.166.219:8080
17:42:20:WU00:FS00:Downloading 116.60KiB
17:42:20:WU00:FS00:Download complete
17:42:20:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:6399 run:134 clone:32 gen:52 core:0xa4 unit:0x000000390002894b5462ccf318518125
17:46:49:WU01:FS00:0xa4:Completed 250000 out of 250000 steps  (100%)
17:46:49:WU01:FS00:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
17:46:59:WU01:FS00:0xa4:
17:46:59:WU01:FS00:0xa4:Finished Work Unit:
17:46:59:WU01:FS00:0xa4:- Reading up to 916296 from "01/wudata_01.trr": Read 916296
17:46:59:WU01:FS00:0xa4:trr file hash check passed.
17:46:59:WU01:FS00:0xa4:- Reading up to 839716 from "01/wudata_01.xtc": Read 839716
17:46:59:WU01:FS00:0xa4:xtc file hash check passed.
17:46:59:WU01:FS00:0xa4:edr file hash check passed.
17:46:59:WU01:FS00:0xa4:logfile size: 24190
17:46:59:WU01:FS00:0xa4:Leaving Run
17:47:01:WU01:FS00:0xa4:- Writing 1782690 bytes of core data to disk...
17:47:01:WU01:FS00:0xa4:Done: 1782178 -> 1722491 (compressed to 96.6 percent)
17:47:01:WU01:FS00:0xa4:  ... Done.
17:47:02:WU01:FS00:0xa4:- Shutting down core
17:47:02:WU01:FS00:0xa4:
17:47:02:WU01:FS00:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
17:47:03:WU01:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
17:47:03:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:9015 run:488 clone:1 gen:197 core:0xa4 unit:0x000000f1664f2de453e55e41f721b973
17:47:03:WU01:FS00:Uploading 1.64MiB to 171.64.65.124
17:47:03:WU01:FS00:Connecting to 171.64.65.124:8080
17:47:03:WU00:FS00:Starting
17:47:03:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/John/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 00 -suffix 01 -version 704 -lifeline 5552 -checkpoint 15 -np 3
17:47:04:WU00:FS00:Started FahCore on PID 7628
17:47:04:WU00:FS00:Core PID:2900
17:47:04:WU00:FS00:FahCore 0xa4 started
17:47:04:WU00:FS00:0xa4:
17:47:04:WU00:FS00:0xa4:*------------------------------*
17:47:04:WU00:FS00:0xa4:Folding@Home Gromacs GB Core
17:47:04:WU00:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
17:47:04:WU00:FS00:0xa4:
17:47:04:WU00:FS00:0xa4:Preparing to commence simulation
17:47:04:WU00:FS00:0xa4:- Looking at optimizations...
17:47:04:WU00:FS00:0xa4:- Created dyn
17:47:04:WU00:FS00:0xa4:- Files status OK
17:47:04:WU00:FS00:0xa4:- Expanded 118884 -> 269764 (decompressed 226.9 percent)
17:47:04:WU00:FS00:0xa4:Called DecompressByteArray: compressed_data_size=118884 data_size=269764, decompressed_data_size=269764 diff=0
17:47:04:WU00:FS00:0xa4:- Digital signature verified
17:47:04:WU00:FS00:0xa4:
17:47:04:WU00:FS00:0xa4:Project: 6399 (Run 134, Clone 32, Gen 52)
17:47:04:WU00:FS00:0xa4:
17:47:04:WU00:FS00:0xa4:Assembly optimizations on if available.
17:47:04:WU00:FS00:0xa4:Entering M.D.
17:47:10:WU00:FS00:0xa4:Mapping NT from 3 to 3 
17:47:10:WU00:FS00:0xa4:Completed 0 out of 5000000 steps  (0%)
17:52:40:WU00:FS00:0xa4:Completed 50000 out of 5000000 steps  (1%)
17:56:51:WARNING:WU01:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
17:56:51:WU01:FS00:Trying to send results to collection server
17:56:51:WU01:FS00:Uploading 1.64MiB to 171.65.103.160
17:56:51:WU01:FS00:Connecting to 171.65.103.160:8080
17:57:58:WU00:FS00:0xa4:Completed 100000 out of 5000000 steps  (2%)
17:58:07:ERROR:WU01:FS00:Exception: 10002: Received short response, expected 512 bytes, got 0
17:58:08:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:9015 run:488 clone:1 gen:197 core:0xa4 unit:0x000000f1664f2de453e55e41f721b973
17:58:08:WU01:FS00:Uploading 1.64MiB to 171.64.65.124
17:58:08:WU01:FS00:Connecting to 171.64.65.124:8080
17:58:09:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
17:58:09:WU01:FS00:Connecting to 171.64.65.124:80
17:58:10:WARNING:WU01:FS00:Exception: Failed to send results to work server: Failed to connect to 171.64.65.124:80: No connection could be made because the target machine actively refused it.
17:58:10:WU01:FS00:Trying to send results to collection server
17:58:10:WU01:FS00:Uploading 1.64MiB to 171.65.103.160
17:58:10:WU01:FS00:Connecting to 171.65.103.160:8080
17:58:15:WU01:FS00:Upload complete
17:58:15:WU01:FS00:Server responded WORK_ACK (400)
17:58:15:WU01:FS00:Final credit estimate, 1030.00 points
17:58:15:WU01:FS00:Cleaning up
18:03:14:WU00:FS00:0xa4:Completed 150000 out of 5000000 steps  (3%)
...
22:49:20:WU00:FS00:0xa4:Completed 3050000 out of 5000000 steps  (61%)
******************************* Date: 2015-04-28 *******************************
22:54:03:WU00:FS00:0xa4:Completed 3100000 out of 5000000 steps  (62%)
...
01:48:30:WU00:FS00:0xa4:Completed 4950000 out of 5000000 steps  (99%)
01:48:32:WU01:FS00:Connecting to 171.67.108.200:8080
01:48:32:WU01:FS00:Assigned to work server 171.64.65.124
01:48:32:WU01:FS00:Requesting new work unit for slot 00: RUNNING cpu:3 from 171.64.65.124
01:48:32:WU01:FS00:Connecting to 171.64.65.124:8080
01:48:33:WU01:FS00:Downloading 807.58KiB
01:48:33:WU01:FS00:Download complete
01:48:34:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:9016 run:405 clone:3 gen:155 core:0xa4 unit:0x000000b4664f2de45491d98025b640f8
01:53:18:WU00:FS00:0xa4:Completed 5000000 out of 5000000 steps  (100%)
01:53:18:WU00:FS00:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
01:53:28:WU00:FS00:0xa4:
01:53:28:WU00:FS00:0xa4:Finished Work Unit:
01:53:28:WU00:FS00:0xa4:- Reading up to 1255848 from "00/wudata_01.trr": Read 1255848
01:53:28:WU00:FS00:0xa4:trr file hash check passed.
01:53:28:WU00:FS00:0xa4:- Reading up to 111152 from "00/wudata_01.xtc": Read 111152
01:53:28:WU00:FS00:0xa4:xtc file hash check passed.
01:53:28:WU00:FS00:0xa4:edr file hash check passed.
01:53:28:WU00:FS00:0xa4:logfile size: 88418
01:53:28:WU00:FS00:0xa4:Leaving Run
01:53:31:WU00:FS00:0xa4:- Writing 1527318 bytes of core data to disk...
01:53:31:WU00:FS00:0xa4:Done: 1526806 -> 1292910 (compressed to 84.6 percent)
01:53:31:WU00:FS00:0xa4:  ... Done.
01:53:32:WU00:FS00:0xa4:- Shutting down core
01:53:32:WU00:FS00:0xa4:
01:53:32:WU00:FS00:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
01:53:32:WU00:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
01:53:32:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:6399 run:134 clone:32 gen:52 core:0xa4 unit:0x000000390002894b5462ccf318518125
01:53:32:WU00:FS00:Uploading 1.23MiB to 155.247.166.219
01:53:32:WU00:FS00:Connecting to 155.247.166.219:8080
01:53:32:WU01:FS00:Starting
01:53:32:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/John/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 704 -lifeline 5552 -checkpoint 15 -np 3
01:53:33:WU01:FS00:Started FahCore on PID 3680
01:53:33:WU01:FS00:Core PID:1580
01:53:33:WU01:FS00:FahCore 0xa4 started
01:53:33:WU01:FS00:0xa4:
01:53:33:WU01:FS00:0xa4:*------------------------------*
01:53:33:WU01:FS00:0xa4:Folding@Home Gromacs GB Core
01:53:33:WU01:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
01:53:33:WU01:FS00:0xa4:
01:53:33:WU01:FS00:0xa4:Preparing to commence simulation
01:53:33:WU01:FS00:0xa4:- Looking at optimizations...
01:53:33:WU01:FS00:0xa4:- Created dyn
01:53:33:WU01:FS00:0xa4:- Files status OK
01:53:33:WU01:FS00:0xa4:- Expanded 826451 -> 1397548 (decompressed 169.1 percent)
01:53:33:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=826451 data_size=1397548, decompressed_data_size=1397548 diff=0
01:53:33:WU01:FS00:0xa4:- Digital signature verified
01:53:33:WU01:FS00:0xa4:
01:53:33:WU01:FS00:0xa4:Project: 9016 (Run 405, Clone 3, Gen 155)
01:53:33:WU01:FS00:0xa4:
01:53:33:WU01:FS00:0xa4:Assembly optimizations on if available.
01:53:33:WU01:FS00:0xa4:Entering M.D.
01:53:38:WU00:FS00:Upload 50.67%
01:53:39:WU01:FS00:0xa4:Mapping NT from 3 to 3 
01:53:40:WU01:FS00:0xa4:Completed 0 out of 250000 steps  (0%)
01:53:43:WU00:FS00:Upload complete
01:53:43:WU00:FS00:Server responded WORK_ACK (400)
01:53:43:WU00:FS00:Final credit estimate, 1376.00 points
01:53:43:WU00:FS00:Cleaning up
01:57:20:WU01:FS00:0xa4:Completed 2500 out of 250000 steps  (1%)
...
04:49:06:WU01:FS00:0xa4:Completed 120000 out of 250000 steps  (48%)
******************************* Date: 2015-04-29 *******************************
04:52:46:WU01:FS00:0xa4:Completed 122500 out of 250000 steps  (49%)
...
08:14:38:WU01:FS00:0xa4:Completed 247500 out of 250000 steps  (99%)
08:14:39:WU02:FS00:Connecting to 171.67.108.200:8080
08:14:39:WU02:FS00:Assigned to work server 155.247.166.219
08:14:39:WU02:FS00:Requesting new work unit for slot 00: RUNNING cpu:3 from 155.247.166.219
08:14:39:WU02:FS00:Connecting to 155.247.166.219:8080
08:14:40:WU02:FS00:Downloading 116.51KiB
08:14:40:WU02:FS00:Download complete
08:14:40:WU02:FS00:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:6399 run:118 clone:36 gen:33 core:0xa4 unit:0x000000250002894b5462ccd1336cfcfa
08:19:27:WU01:FS00:0xa4:Completed 250000 out of 250000 steps  (100%)
08:19:28:WU01:FS00:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
08:19:38:WU01:FS00:0xa4:
08:19:38:WU01:FS00:0xa4:Finished Work Unit:
08:19:38:WU01:FS00:0xa4:- Reading up to 811488 from "01/wudata_01.trr": Read 811488
08:19:38:WU01:FS00:0xa4:trr file hash check passed.
08:19:38:WU01:FS00:0xa4:- Reading up to 746380 from "01/wudata_01.xtc": Read 746380
08:19:38:WU01:FS00:0xa4:xtc file hash check passed.
08:19:38:WU01:FS00:0xa4:edr file hash check passed.
08:19:38:WU01:FS00:0xa4:logfile size: 24002
08:19:38:WU01:FS00:0xa4:Leaving Run
08:19:43:WU01:FS00:0xa4:- Writing 1584358 bytes of core data to disk...
08:19:44:WU01:FS00:0xa4:Done: 1583846 -> 1538190 (compressed to 97.1 percent)
08:19:44:WU01:FS00:0xa4:  ... Done.
08:19:45:WU01:FS00:0xa4:- Shutting down core
08:19:45:WU01:FS00:0xa4:
08:19:45:WU01:FS00:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
08:19:46:WU01:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
08:19:46:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:9016 run:405 clone:3 gen:155 core:0xa4 unit:0x000000b4664f2de45491d98025b640f8
08:19:46:WU01:FS00:Uploading 1.47MiB to 171.64.65.124
08:19:46:WU01:FS00:Connecting to 171.64.65.124:8080
08:19:46:WU02:FS00:Starting
08:19:46:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/John/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 02 -suffix 01 -version 704 -lifeline 5552 -checkpoint 15 -np 3
08:19:48:WU02:FS00:Started FahCore on PID 2600
08:19:48:WU02:FS00:Core PID:792
08:19:48:WU02:FS00:FahCore 0xa4 started
08:19:49:WU02:FS00:0xa4:
08:19:49:WU02:FS00:0xa4:*------------------------------*
08:19:49:WU02:FS00:0xa4:Folding@Home Gromacs GB Core
08:19:49:WU02:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
08:19:49:WU02:FS00:0xa4:
08:19:49:WU02:FS00:0xa4:Preparing to commence simulation
08:19:49:WU02:FS00:0xa4:- Looking at optimizations...
08:19:49:WU02:FS00:0xa4:- Created dyn
08:19:49:WU02:FS00:0xa4:- Files status OK
08:19:49:WU02:FS00:0xa4:- Expanded 118792 -> 269764 (decompressed 227.0 percent)
08:19:49:WU02:FS00:0xa4:Called DecompressByteArray: compressed_data_size=118792 data_size=269764, decompressed_data_size=269764 diff=0
08:19:49:WU02:FS00:0xa4:- Digital signature verified
08:19:49:WU02:FS00:0xa4:
08:19:49:WU02:FS00:0xa4:Project: 6399 (Run 118, Clone 36, Gen 33)
08:19:49:WU02:FS00:0xa4:
08:19:49:WU02:FS00:0xa4:Assembly optimizations on if available.
08:19:49:WU02:FS00:0xa4:Entering M.D.
08:19:55:WU02:FS00:0xa4:Mapping NT from 3 to 3 
08:19:55:WU02:FS00:0xa4:Completed 0 out of 5000000 steps  (0%)
08:27:09:WU02:FS00:0xa4:Completed 50000 out of 5000000 steps  (1%)
...
10:46:16:WU02:FS00:0xa4:Completed 1300000 out of 5000000 steps  (26%)
******************************* Date: 2015-04-29 *******************************
10:51:10:WU02:FS00:0xa4:Completed 1350000 out of 5000000 steps  (27%)
10:56:00:WU02:FS00:0xa4:Completed 1400000 out of 5000000 steps  (28%)
11:00:59:WU02:FS00:0xa4:Completed 1450000 out of 5000000 steps  (29%)
11:06:26:WU02:FS00:0xa4:Completed 1500000 out of 5000000 steps  (30%)
11:11:20:WU02:FS00:0xa4:Completed 1550000 out of 5000000 steps  (31%)
11:16:08:WU02:FS00:0xa4:Completed 1600000 out of 5000000 steps  (32%)
11:21:03:WU02:FS00:0xa4:Completed 1650000 out of 5000000 steps  (33%)
11:27:53:WU02:FS00:0xa4:Completed 1700000 out of 5000000 steps  (34%)
11:43:42:WU02:FS00:0xa4:Completed 1750000 out of 5000000 steps  (35%)
11:56:48:WU02:FS00:0xa4:Completed 1800000 out of 5000000 steps  (36%)
12:02:26:WU02:FS00:0xa4:Completed 1850000 out of 5000000 steps  (37%)
12:13:40:WU02:FS00:0xa4:Completed 1900000 out of 5000000 steps  (38%)
12:30:56:WU02:FS00:0xa4:Completed 1950000 out of 5000000 steps  (39%)
12:35:48:WU02:FS00:0xa4:Completed 2000000 out of 5000000 steps  (40%)
12:40:42:WU02:FS00:0xa4:Completed 2050000 out of 5000000 steps  (41%)
12:45:37:WU02:FS00:0xa4:Completed 2100000 out of 5000000 steps  (42%)
12:50:29:WU02:FS00:0xa4:Completed 2150000 out of 5000000 steps  (43%)
12:55:21:WU02:FS00:0xa4:Completed 2200000 out of 5000000 steps  (44%)
13:00:17:WU02:FS00:0xa4:Completed 2250000 out of 5000000 steps  (45%)
13:05:12:WU02:FS00:0xa4:Completed 2300000 out of 5000000 steps  (46%)
13:10:10:WU02:FS00:0xa4:Completed 2350000 out of 5000000 steps  (47%)
13:15:02:WU02:FS00:0xa4:Completed 2400000 out of 5000000 steps  (48%)
13:19:55:WU02:FS00:0xa4:Completed 2450000 out of 5000000 steps  (49%)
13:24:45:WU02:FS00:0xa4:Completed 2500000 out of 5000000 steps  (50%)
13:29:37:WU02:FS00:0xa4:Completed 2550000 out of 5000000 steps  (51%)
13:34:28:WU02:FS00:0xa4:Completed 2600000 out of 5000000 steps  (52%)
13:39:19:WU02:FS00:0xa4:Completed 2650000 out of 5000000 steps  (53%)
13:44:13:WU02:FS00:0xa4:Completed 2700000 out of 5000000 steps  (54%)
13:49:08:WU02:FS00:0xa4:Completed 2750000 out of 5000000 steps  (55%)
13:54:02:WU02:FS00:0xa4:Completed 2800000 out of 5000000 steps  (56%)
13:58:55:WU02:FS00:0xa4:Completed 2850000 out of 5000000 steps  (57%)
14:03:46:WU02:FS00:0xa4:Completed 2900000 out of 5000000 steps  (58%)
14:08:36:WU02:FS00:0xa4:Completed 2950000 out of 5000000 steps  (59%)
14:13:29:WU02:FS00:0xa4:Completed 3000000 out of 5000000 steps  (60%)
14:18:21:WU02:FS00:0xa4:Completed 3050000 out of 5000000 steps  (61%)
14:23:12:WU02:FS00:0xa4:Completed 3100000 out of 5000000 steps  (62%)
14:28:06:WU02:FS00:0xa4:Completed 3150000 out of 5000000 steps  (63%)
14:33:06:WU02:FS00:0xa4:Completed 3200000 out of 5000000 steps  (64%)
14:38:02:WU02:FS00:0xa4:Completed 3250000 out of 5000000 steps  (65%)
14:42:59:WU02:FS00:0xa4:Completed 3300000 out of 5000000 steps  (66%)
14:47:59:WU02:FS00:0xa4:Completed 3350000 out of 5000000 steps  (67%)
14:52:57:WU02:FS00:0xa4:Completed 3400000 out of 5000000 steps  (68%)
14:57:56:WU02:FS00:0xa4:Completed 3450000 out of 5000000 steps  (69%)
15:02:57:WU02:FS00:0xa4:Completed 3500000 out of 5000000 steps  (70%)
15:07:55:WU02:FS00:0xa4:Completed 3550000 out of 5000000 steps  (71%)
15:12:52:WU02:FS00:0xa4:Completed 3600000 out of 5000000 steps  (72%)
15:17:48:WU02:FS00:0xa4:Completed 3650000 out of 5000000 steps  (73%)
15:22:45:WU02:FS00:0xa4:Completed 3700000 out of 5000000 steps  (74%)
15:27:41:WU02:FS00:0xa4:Completed 3750000 out of 5000000 steps  (75%)
15:32:41:WU02:FS00:0xa4:Completed 3800000 out of 5000000 steps  (76%)
15:37:37:WU02:FS00:0xa4:Completed 3850000 out of 5000000 steps  (77%)
15:42:34:WU02:FS00:0xa4:Completed 3900000 out of 5000000 steps  (78%)
15:47:33:WU02:FS00:0xa4:Completed 3950000 out of 5000000 steps  (79%)
15:52:31:WU02:FS00:0xa4:Completed 4000000 out of 5000000 steps  (80%)
15:57:33:WU02:FS00:0xa4:Completed 4050000 out of 5000000 steps  (81%)
16:02:56:WU02:FS00:0xa4:Completed 4100000 out of 5000000 steps  (82%)
16:04:07:FS00:Paused
16:04:07:FS00:Shutting core down
16:04:13:WU02:FS00:0xa4:Client no longer detected. Shutting down core 
16:04:13:WU02:FS00:0xa4:
16:04:13:WU02:FS00:0xa4:Folding@home Core Shutdown: CLIENT_DIED
16:04:13:WU02:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
16:23:29:FS00:Unpaused
16:23:29:WU02:FS00:Starting
16:23:29:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/John/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 02 -suffix 01 -version 704 -lifeline 5552 -checkpoint 15 -np 3
16:23:29:WU02:FS00:Started FahCore on PID 1596
16:23:29:WU02:FS00:Core PID:4480
16:23:29:WU02:FS00:FahCore 0xa4 started
16:23:30:WU02:FS00:0xa4:
16:23:30:WU02:FS00:0xa4:*------------------------------*
16:23:30:WU02:FS00:0xa4:Folding@Home Gromacs GB Core
16:23:30:WU02:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
16:23:30:WU02:FS00:0xa4:
16:23:30:WU02:FS00:0xa4:Preparing to commence simulation
16:23:30:WU02:FS00:0xa4:- Looking at optimizations...
16:23:30:WU02:FS00:0xa4:- Files status OK
16:23:30:WU02:FS00:0xa4:- Expanded 118792 -> 269764 (decompressed 227.0 percent)
16:23:30:WU02:FS00:0xa4:Called DecompressByteArray: compressed_data_size=118792 data_size=269764, decompressed_data_size=269764 diff=0
16:23:30:WU02:FS00:0xa4:- Digital signature verified
16:23:30:WU02:FS00:0xa4:
16:23:30:WU02:FS00:0xa4:Project: 6399 (Run 118, Clone 36, Gen 33)
16:23:30:WU02:FS00:0xa4:
16:23:30:WU02:FS00:0xa4:Assembly optimizations on if available.
16:23:30:WU02:FS00:0xa4:Entering M.D.
16:23:35:WU02:FS00:0xa4:Using Gromacs checkpoints
16:23:35:WU02:FS00:0xa4:Mapping NT from 3 to 3 
16:23:36:WU02:FS00:0xa4:Resuming from checkpoint
16:23:36:WU02:FS00:0xa4:Verified 02/wudata_01.log
16:23:36:WU02:FS00:0xa4:Verified 02/wudata_01.trr
16:23:36:WU02:FS00:0xa4:Verified 02/wudata_01.xtc
16:23:36:WU02:FS00:0xa4:Verified 02/wudata_01.edr
16:23:36:WU02:FS00:0xa4:Completed 3973700 out of 5000000 steps  (79%)
16:26:11:WU02:FS00:0xa4:Completed 4000000 out of 5000000 steps  (80%)
16:31:21:WU02:FS00:0xa4:Completed 4050000 out of 5000000 steps  (81%)
16:36:19:WU02:FS00:0xa4:Completed 4100000 out of 5000000 steps  (82%)
16:41:18:WU02:FS00:0xa4:Completed 4150000 out of 5000000 steps  (83%)
16:46:28:WU02:FS00:0xa4:Completed 4200000 out of 5000000 steps  (84%)
******************************* Date: 2015-04-29 *******************************
16:51:25:WU02:FS00:0xa4:Completed 4250000 out of 5000000 steps  (85%)
16:56:21:WU02:FS00:0xa4:Completed 4300000 out of 5000000 steps  (86%)
17:01:23:WU02:FS00:0xa4:Completed 4350000 out of 5000000 steps  (87%)
17:06:46:WU02:FS00:0xa4:Completed 4400000 out of 5000000 steps  (88%)
17:11:49:WU02:FS00:0xa4:Completed 4450000 out of 5000000 steps  (89%)
17:16:46:WU02:FS00:0xa4:Completed 4500000 out of 5000000 steps  (90%)
17:21:44:WU02:FS00:0xa4:Completed 4550000 out of 5000000 steps  (91%)
17:26:41:WU02:FS00:0xa4:Completed 4600000 out of 5000000 steps  (92%)
17:31:37:WU02:FS00:0xa4:Completed 4650000 out of 5000000 steps  (93%)
17:36:35:WU02:FS00:0xa4:Completed 4700000 out of 5000000 steps  (94%)
17:41:32:WU02:FS00:0xa4:Completed 4750000 out of 5000000 steps  (95%)
17:46:29:WU02:FS00:0xa4:Completed 4800000 out of 5000000 steps  (96%)
17:51:26:WU02:FS00:0xa4:Completed 4850000 out of 5000000 steps  (97%)
17:56:22:WU02:FS00:0xa4:Completed 4900000 out of 5000000 steps  (98%)
18:01:21:WU02:FS00:0xa4:Completed 4950000 out of 5000000 steps  (99%)
18:01:21:WU00:FS00:Connecting to 171.67.108.200:8080
18:01:22:WU00:FS00:Assigned to work server 171.64.65.124
18:01:22:WU00:FS00:Requesting new work unit for slot 00: RUNNING cpu:3 from 171.64.65.124
18:01:22:WU00:FS00:Connecting to 171.64.65.124:8080
18:01:22:WU00:FS00:Downloading 902.35KiB
18:01:23:WU00:FS00:Download complete
18:01:23:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:9014 run:204 clone:0 gen:232 core:0xa4 unit:0x0000011b664f2de453e558549708f10f
18:06:18:WU02:FS00:0xa4:Completed 5000000 out of 5000000 steps  (100%)
18:06:18:WU02:FS00:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
18:06:28:WU02:FS00:0xa4:
18:06:28:WU02:FS00:0xa4:Finished Work Unit:
18:06:28:WU02:FS00:0xa4:- Reading up to 1255848 from "02/wudata_01.trr": Read 1255848
18:06:28:WU02:FS00:0xa4:trr file hash check passed.
18:06:28:WU02:FS00:0xa4:- Reading up to 109868 from "02/wudata_01.xtc": Read 109868
18:06:28:WU02:FS00:0xa4:xtc file hash check passed.
18:06:28:WU02:FS00:0xa4:edr file hash check passed.
18:06:28:WU02:FS00:0xa4:logfile size: 89188
18:06:28:WU02:FS00:0xa4:Leaving Run
18:06:29:WU02:FS00:0xa4:- Writing 1526804 bytes of core data to disk...
18:06:29:WU02:FS00:0xa4:Done: 1526292 -> 1291073 (compressed to 84.5 percent)
18:06:29:WU02:FS00:0xa4:  ... Done.
18:06:30:WU02:FS00:0xa4:- Shutting down core
18:06:30:WU02:FS00:0xa4:
18:06:30:WU02:FS00:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
18:06:30:WU02:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
18:06:30:WU02:FS00:Sending unit results: id:02 state:SEND error:NO_ERROR project:6399 run:118 clone:36 gen:33 core:0xa4 unit:0x000000250002894b5462ccd1336cfcfa
18:06:30:WU02:FS00:Uploading 1.23MiB to 155.247.166.219
18:06:30:WU02:FS00:Connecting to 155.247.166.219:8080
18:06:30:WU00:FS00:Starting
18:06:30:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/John/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 00 -suffix 01 -version 704 -lifeline 5552 -checkpoint 15 -np 3
18:06:30:WU00:FS00:Started FahCore on PID 8184
18:06:31:WU00:FS00:Core PID:3404
18:06:31:WU00:FS00:FahCore 0xa4 started
18:06:31:WU00:FS00:0xa4:
18:06:31:WU00:FS00:0xa4:*------------------------------*
18:06:31:WU00:FS00:0xa4:Folding@Home Gromacs GB Core
18:06:31:WU00:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
18:06:31:WU00:FS00:0xa4:
18:06:31:WU00:FS00:0xa4:Preparing to commence simulation
18:06:31:WU00:FS00:0xa4:- Looking at optimizations...
18:06:31:WU00:FS00:0xa4:- Created dyn
18:06:31:WU00:FS00:0xa4:- Files status OK
18:06:31:WU00:FS00:0xa4:- Expanded 923495 -> 1534204 (decompressed 166.1 percent)
18:06:31:WU00:FS00:0xa4:Called DecompressByteArray: compressed_data_size=923495 data_size=1534204, decompressed_data_size=1534204 diff=0
18:06:31:WU00:FS00:0xa4:- Digital signature verified
18:06:31:WU00:FS00:0xa4:
18:06:31:WU00:FS00:0xa4:Project: 9014 (Run 204, Clone 0, Gen 232)
18:06:31:WU00:FS00:0xa4:
18:06:31:WU00:FS00:0xa4:Assembly optimizations on if available.
18:06:31:WU00:FS00:0xa4:Entering M.D.
18:06:36:WU02:FS00:Upload 55.81%
18:06:37:WU00:FS00:0xa4:Mapping NT from 3 to 3 
18:06:38:WU00:FS00:0xa4:Completed 0 out of 250000 steps  (0%)
18:06:41:WU02:FS00:Upload complete
18:06:41:WU02:FS00:Server responded WORK_ACK (400)
18:06:41:WU02:FS00:Final credit estimate, 1253.00 points
18:06:41:WU02:FS00:Cleaning up
18:10:45:WU00:FS00:0xa4:Completed 2500 out of 250000 steps  (1%)
18:14:51:WU00:FS00:0xa4:Completed 5000 out of 250000 steps  (2%)
18:18:57:WU00:FS00:0xa4:Completed 7500 out of 250000 steps  (3%)
18:23:01:WU00:FS00:0xa4:Completed 10000 out of 250000 steps  (4%)
18:27:04:WU00:FS00:0xa4:Completed 12500 out of 250000 steps  (5%)
18:31:10:WU00:FS00:0xa4:Completed 15000 out of 250000 steps  (6%)
18:35:15:WU00:FS00:0xa4:Completed 17500 out of 250000 steps  (7%)
18:39:20:WU00:FS00:0xa4:Completed 20000 out of 250000 steps  (8%)
18:43:25:WU00:FS00:0xa4:Completed 22500 out of 250000 steps  (9%)
18:47:30:WU00:FS00:0xa4:Completed 25000 out of 250000 steps  (10%)
18:51:36:WU00:FS00:0xa4:Completed 27500 out of 250000 steps  (11%)
18:56:13:WU00:FS00:0xa4:Completed 30000 out of 250000 steps  (12%)
19:00:23:WU00:FS00:0xa4:Completed 32500 out of 250000 steps  (13%)
19:04:30:WU00:FS00:0xa4:Completed 35000 out of 250000 steps  (14%)
19:08:37:WU00:FS00:0xa4:Completed 37500 out of 250000 steps  (15%)
19:12:43:WU00:FS00:0xa4:Completed 40000 out of 250000 steps  (16%)
19:16:52:WU00:FS00:0xa4:Completed 42500 out of 250000 steps  (17%)
19:20:59:WU00:FS00:0xa4:Completed 45000 out of 250000 steps  (18%)
19:25:07:WU00:FS00:0xa4:Completed 47500 out of 250000 steps  (19%)
19:29:10:WU00:FS00:0xa4:Completed 50000 out of 250000 steps  (20%)
19:33:17:WU00:FS00:0xa4:Completed 52500 out of 250000 steps  (21%)
19:37:25:WU00:FS00:0xa4:Completed 55000 out of 250000 steps  (22%)
19:41:31:WU00:FS00:0xa4:Completed 57500 out of 250000 steps  (23%)
19:45:36:WU00:FS00:0xa4:Completed 60000 out of 250000 steps  (24%)
19:49:41:WU00:FS00:0xa4:Completed 62500 out of 250000 steps  (25%)
19:53:46:WU00:FS00:0xa4:Completed 65000 out of 250000 steps  (26%)
19:57:50:WU00:FS00:0xa4:Completed 67500 out of 250000 steps  (27%)
20:01:58:WU00:FS00:0xa4:Completed 70000 out of 250000 steps  (28%)
20:06:05:WU00:FS00:0xa4:Completed 72500 out of 250000 steps  (29%)
20:10:10:WU00:FS00:0xa4:Completed 75000 out of 250000 steps  (30%)
20:14:13:WU00:FS00:0xa4:Completed 77500 out of 250000 steps  (31%)
20:18:19:WU00:FS00:0xa4:Completed 80000 out of 250000 steps  (32%)
20:22:25:WU00:FS00:0xa4:Completed 82500 out of 250000 steps  (33%)
20:26:30:WU00:FS00:0xa4:Completed 85000 out of 250000 steps  (34%)
20:30:35:WU00:FS00:0xa4:Completed 87500 out of 250000 steps  (35%)
20:34:39:WU00:FS00:0xa4:Completed 90000 out of 250000 steps  (36%)
20:38:43:WU00:FS00:0xa4:Completed 92500 out of 250000 steps  (37%)
20:42:47:WU00:FS00:0xa4:Completed 95000 out of 250000 steps  (38%)
20:46:55:WU00:FS00:0xa4:Completed 97500 out of 250000 steps  (39%)
20:51:00:WU00:FS00:0xa4:Completed 100000 out of 250000 steps  (40%)
20:55:07:WU00:FS00:0xa4:Completed 102500 out of 250000 steps  (41%)
20:59:12:WU00:FS00:0xa4:Completed 105000 out of 250000 steps  (42%)
21:03:18:WU00:FS00:0xa4:Completed 107500 out of 250000 steps  (43%)
21:07:25:WU00:FS00:0xa4:Completed 110000 out of 250000 steps  (44%)
21:11:32:WU00:FS00:0xa4:Completed 112500 out of 250000 steps  (45%)
21:15:37:WU00:FS00:0xa4:Completed 115000 out of 250000 steps  (46%)
21:19:43:WU00:FS00:0xa4:Completed 117500 out of 250000 steps  (47%)
21:23:47:WU00:FS00:0xa4:Completed 120000 out of 250000 steps  (48%)
21:27:51:WU00:FS00:0xa4:Completed 122500 out of 250000 steps  (49%)
21:31:59:WU00:FS00:0xa4:Completed 125000 out of 250000 steps  (50%)
21:36:03:WU00:FS00:0xa4:Completed 127500 out of 250000 steps  (51%)
21:40:09:WU00:FS00:0xa4:Completed 130000 out of 250000 steps  (52%)
21:44:15:WU00:FS00:0xa4:Completed 132500 out of 250000 steps  (53%)
21:48:22:WU00:FS00:0xa4:Completed 135000 out of 250000 steps  (54%)
21:52:29:WU00:FS00:0xa4:Completed 137500 out of 250000 steps  (55%)
21:56:36:WU00:FS00:0xa4:Completed 140000 out of 250000 steps  (56%)
22:00:43:WU00:FS00:0xa4:Completed 142500 out of 250000 steps  (57%)
22:04:50:WU00:FS00:0xa4:Completed 145000 out of 250000 steps  (58%)
22:08:55:WU00:FS00:0xa4:Completed 147500 out of 250000 steps  (59%)
22:13:06:WU00:FS00:0xa4:Completed 150000 out of 250000 steps  (60%)
22:17:12:WU00:FS00:0xa4:Completed 152500 out of 250000 steps  (61%)
22:21:20:WU00:FS00:0xa4:Completed 155000 out of 250000 steps  (62%)
22:25:26:WU00:FS00:0xa4:Completed 157500 out of 250000 steps  (63%)
22:29:30:WU00:FS00:0xa4:Completed 160000 out of 250000 steps  (64%)
22:33:40:WU00:FS00:0xa4:Completed 162500 out of 250000 steps  (65%)
22:37:46:WU00:FS00:0xa4:Completed 165000 out of 250000 steps  (66%)
22:41:52:WU00:FS00:0xa4:Completed 167500 out of 250000 steps  (67%)
22:46:00:WU00:FS00:0xa4:Completed 170000 out of 250000 steps  (68%)
22:50:04:WU00:FS00:0xa4:Completed 172500 out of 250000 steps  (69%)
******************************* Date: 2015-04-29 *******************************
22:54:11:WU00:FS00:0xa4:Completed 175000 out of 250000 steps  (70%)
22:58:19:WU00:FS00:0xa4:Completed 177500 out of 250000 steps  (71%)
23:02:28:WU00:FS00:0xa4:Completed 180000 out of 250000 steps  (72%)
23:07:08:WU00:FS00:0xa4:Completed 182500 out of 250000 steps  (73%)
23:11:15:WU00:FS00:0xa4:Completed 185000 out of 250000 steps  (74%)
23:15:20:WU00:FS00:0xa4:Completed 187500 out of 250000 steps  (75%)
23:19:27:WU00:FS00:0xa4:Completed 190000 out of 250000 steps  (76%)
23:23:35:WU00:FS00:0xa4:Completed 192500 out of 250000 steps  (77%)
23:27:40:WU00:FS00:0xa4:Completed 195000 out of 250000 steps  (78%)
23:31:47:WU00:FS00:0xa4:Completed 197500 out of 250000 steps  (79%)
23:35:54:WU00:FS00:0xa4:Completed 200000 out of 250000 steps  (80%)
23:39:59:WU00:FS00:0xa4:Completed 202500 out of 250000 steps  (81%)
23:44:05:WU00:FS00:0xa4:Completed 205000 out of 250000 steps  (82%)
23:48:11:WU00:FS00:0xa4:Completed 207500 out of 250000 steps  (83%)
23:52:17:WU00:FS00:0xa4:Completed 210000 out of 250000 steps  (84%)
23:56:22:WU00:FS00:0xa4:Completed 212500 out of 250000 steps  (85%)
00:00:30:WU00:FS00:0xa4:Completed 215000 out of 250000 steps  (86%)
00:04:39:WU00:FS00:0xa4:Completed 217500 out of 250000 steps  (87%)
00:08:44:WU00:FS00:0xa4:Completed 220000 out of 250000 steps  (88%)
00:12:51:WU00:FS00:0xa4:Completed 222500 out of 250000 steps  (89%)
00:16:59:WU00:FS00:0xa4:Completed 225000 out of 250000 steps  (90%)
00:21:08:WU00:FS00:0xa4:Completed 227500 out of 250000 steps  (91%)
00:25:18:WU00:FS00:0xa4:Completed 230000 out of 250000 steps  (92%)
00:29:25:WU00:FS00:0xa4:Completed 232500 out of 250000 steps  (93%)
00:33:35:WU00:FS00:0xa4:Completed 235000 out of 250000 steps  (94%)
00:37:42:WU00:FS00:0xa4:Completed 237500 out of 250000 steps  (95%)
00:41:51:WU00:FS00:0xa4:Completed 240000 out of 250000 steps  (96%)
00:46:00:WU00:FS00:0xa4:Completed 242500 out of 250000 steps  (97%)
00:50:07:WU00:FS00:0xa4:Completed 245000 out of 250000 steps  (98%)
00:54:15:WU00:FS00:0xa4:Completed 247500 out of 250000 steps  (99%)
00:54:16:WU03:FS00:Connecting to 171.67.108.200:8080
00:54:17:WU03:FS00:Assigned to work server 171.64.65.124
00:54:17:WU03:FS00:Requesting new work unit for slot 00: RUNNING cpu:3 from 171.64.65.124
00:54:17:WU03:FS00:Connecting to 171.64.65.124:8080
00:54:17:WU03:FS00:Downloading 901.86KiB
00:54:18:WU03:FS00:Download complete
00:54:18:WU03:FS00:Received Unit: id:03 state:DOWNLOAD error:NO_ERROR project:9015 run:452 clone:1 gen:188 core:0xa4 unit:0x000000e4664f2de453e55e04c5894668
00:58:21:WU00:FS00:0xa4:Completed 250000 out of 250000 steps  (100%)
00:58:22:WU00:FS00:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
00:58:32:WU00:FS00:0xa4:
00:58:32:WU00:FS00:0xa4:Finished Work Unit:
00:58:32:WU00:FS00:0xa4:- Reading up to 915792 from "00/wudata_01.trr": Read 915792
00:58:32:WU00:FS00:0xa4:trr file hash check passed.
00:58:32:WU00:FS00:0xa4:- Reading up to 839324 from "00/wudata_01.xtc": Read 839324
00:58:32:WU00:FS00:0xa4:xtc file hash check passed.
00:58:32:WU00:FS00:0xa4:edr file hash check passed.
00:58:32:WU00:FS00:0xa4:logfile size: 24120
00:58:32:WU00:FS00:0xa4:Leaving Run
00:58:33:WU00:FS00:0xa4:- Writing 1781724 bytes of core data to disk...
00:58:34:WU00:FS00:0xa4:Done: 1781212 -> 1721745 (compressed to 96.6 percent)
00:58:34:WU00:FS00:0xa4:  ... Done.
00:58:35:WU00:FS00:0xa4:- Shutting down core
00:58:35:WU00:FS00:0xa4:
00:58:35:WU00:FS00:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
00:58:36:WU00:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
00:58:36:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:9014 run:204 clone:0 gen:232 core:0xa4 unit:0x0000011b664f2de453e558549708f10f
00:58:36:WU00:FS00:Uploading 1.64MiB to 171.64.65.124
00:58:36:WU00:FS00:Connecting to 171.64.65.124:8080
00:58:36:WU03:FS00:Starting
00:58:36:WU03:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/John/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 03 -suffix 01 -version 704 -lifeline 5552 -checkpoint 15 -np 3
00:58:36:WU03:FS00:Started FahCore on PID 3616
00:58:37:WU03:FS00:Core PID:5692
00:58:37:WU03:FS00:FahCore 0xa4 started
00:58:37:WU03:FS00:0xa4:
00:58:37:WU03:FS00:0xa4:*------------------------------*
00:58:37:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
00:58:37:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
00:58:37:WU03:FS00:0xa4:
00:58:37:WU03:FS00:0xa4:Preparing to commence simulation
00:58:37:WU03:FS00:0xa4:- Looking at optimizations...
00:58:37:WU03:FS00:0xa4:- Created dyn
00:58:37:WU03:FS00:0xa4:- Files status OK
00:58:37:WU03:FS00:0xa4:- Expanded 922991 -> 1526808 (decompressed 165.4 percent)
00:58:37:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=922991 data_size=1526808, decompressed_data_size=1526808 diff=0
00:58:37:WU03:FS00:0xa4:- Digital signature verified
00:58:37:WU03:FS00:0xa4:
00:58:37:WU03:FS00:0xa4:Project: 9015 (Run 452, Clone 1, Gen 188)
00:58:37:WU03:FS00:0xa4:
00:58:37:WU03:FS00:0xa4:Assembly optimizations on if available.
00:58:37:WU03:FS00:0xa4:Entering M.D.
00:58:40:WU00:FS00:Upload complete
00:58:40:WU00:FS00:Server responded WORK_ACK (400)
00:58:40:WU00:FS00:Final credit estimate, 1043.00 points
00:58:41:WU00:FS00:Cleaning up
00:58:43:WU03:FS00:0xa4:Mapping NT from 3 to 3 
00:58:43:WU03:FS00:0xa4:Completed 0 out of 250000 steps  (0%)
01:02:54:WU03:FS00:0xa4:Completed 2500 out of 250000 steps  (1%)
01:07:08:WU03:FS00:0xa4:Completed 5000 out of 250000 steps  (2%)
01:11:20:WU03:FS00:0xa4:Completed 7500 out of 250000 steps  (3%)
Mod edit: Added code tags.
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: WU 9016 stuck at "Sending"

Post by bruce »

Have you restarted FAHClient (perhaps by rebooting)? Certain types of network errors are known to hang communications. It's entirely possible that the client supports two Up streams and only one is hung.
Donk_Enterprises
Posts: 4
Joined: Sat May 02, 2015 5:13 am

Re: WU 9016 stuck at "Sending"

Post by Donk_Enterprises »

Rebooting got rid of the stuck upload, but I do not know if it got posted.

I wonder if I caused the problem in the first place by monkeying around with the "Folding Power" slider? I recently noticed that the logfile warns about possible problems if that setting is changed while a WU is in progress since the number of CPU cores available is changed.
Joe_H
Site Admin
Posts: 7870
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: WU 9016 stuck at "Sending"

Post by Joe_H »

The stuck WU did get sent and accepted:
Hi Donk_Enterprises (team 0),
Your WU (P9016 R405 C3 G155) was added to the stats database on 2015-05-03 04:08:46 for 207 points of credit.
Changing the Folding Power slider probably had nothing to do with this WU getting stuck. That mention of possible problems normally only applies to changes in core count that might be seen on a system with more cores available such as an i7. There changing from 8 cores to 7 might cause a failure.

One thing the additional parts of the log did show was that you also nearly had a WU get stuck uploading earlier in the day on the 28th. It took several retries, and the WU did upload eventually to the CS instead of the WS after about 11 minutes. So if there were some network connection problems then, they may have cleared up since.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Post Reply