Collection Server 128.252.203.2 failures

Moderators: Site Moderators, FAHC Science Team

Post Reply
Aurum
Posts: 296
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Collection Server 128.252.203.2 failures

Post by Aurum »

I'm getting a lot of failed uploads to collection server 128.252.203.2 :?:

Code: Select all

23:26:37:WARNING:WU06:FS01:Exception: Failed to send results to work server: Transfer failed
23:26:37:WU06:FS01:Trying to send results to collection server
23:26:37:WU06:FS01:Uploading 21.89MiB to 128.252.203.2
23:26:37:WU06:FS01:Connecting to 128.252.203.2:8080
In Science We Trust Image
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Collection Server 128.252.203.2 failures

Post by bruce »

The information you posted doesn't indicate the collection server has failed. The messages BEFORE that segment should be about the Work Server, which apparently failed. The messages AFTER that segment should be about the Collection server at 128.252.203.2.

Please be a little more verbose when you post a log (and include the Project/Run/Clone/Gen numbers of the WU being discussed.)

See the instructions in the signature block of this post.
Aurum
Posts: 296
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: Collection Server 128.252.203.2 failures

Post by Aurum »

Code: Select all

*********************** Log Started 2017-04-09T02:30:07Z ***********************
02:30:07:************************* Folding@home Client *************************
02:30:07:        Website: http://folding.stanford.edu/
02:30:07:      Copyright: (c) 2009-2016 Stanford University
02:30:07:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
02:30:07:           Args: 
02:30:07:         Config: C:\Users\Rig-9\AppData\Roaming\FAHClient\config.xml
02:30:07:******************************** Build ********************************
02:30:07:        Version: 7.4.15
02:30:07:           Date: Aug 17 2016
02:30:07:           Time: 04:33:41
02:30:07:     Repository: Git
02:30:07:       Revision: 4f3e0e25571a9f691719f0c273739294bde517dd
02:30:07:         Branch: master
02:30:07:       Compiler: GNU 5.3.1 20160205
02:30:07:        Options: -std=gnu++98 -I/mingw64/include -O3 -funroll-loops -ffast-math
02:30:07:                 -mfpmath=sse -fno-unsafe-math-optimizations -msse2
02:30:07:       Platform: linux2 4.6.0-1-amd64
02:30:07:           Bits: 64
02:30:07:           Mode: Release
02:30:07:******************************* System ********************************
02:30:07:            CPU: Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz
02:30:07:         CPU ID: GenuineIntel Family 6 Model 63 Stepping 2
02:30:07:           CPUs: 24
02:30:07:         Memory: 15.84GiB
02:30:07:    Free Memory: 14.35GiB
02:30:07:        Threads: WINDOWS_THREADS
02:30:07:     OS Version: 6.1
02:30:07:    Has Battery: false
02:30:07:     On Battery: false
02:30:07:     UTC Offset: -7
02:30:07:            PID: 2956
02:30:07:            CWD: C:\Users\Rig-9\AppData\Roaming\FAHClient
02:30:07:             OS: Windows 7 Home Premium Service Pack 1
02:30:07:        OS Arch: AMD64
02:30:07:           GPUs: 4
02:30:07:          GPU 0: Bus:2 Slot:0 NVIDIA:5 GP104 [GeForce GTX 1070]
02:30:07:          GPU 1: Bus:2 Slot:0 NVIDIA:5 GP104 [GeForce GTX 1070]
02:30:07:          GPU 2: Bus:2 Slot:0 NVIDIA:5 GP104 [GeForce GTX 1070]
02:30:07:          GPU 3: Bus:2 Slot:0 NVIDIA:5 GP104 [GeForce GTX 1070]
02:30:07:  CUDA Device 0: Platform:0 Device:0 Bus:2 Slot:0 Compute:6.1 Driver:8.0
02:30:07:  CUDA Device 1: Platform:0 Device:1 Bus:3 Slot:0 Compute:6.1 Driver:8.0
02:30:07:  CUDA Device 2: Platform:0 Device:2 Bus:4 Slot:0 Compute:6.1 Driver:8.0
02:30:07:  CUDA Device 3: Platform:0 Device:3 Bus:1 Slot:0 Compute:6.1 Driver:8.0
02:30:07:OpenCL Device 0: Platform:0 Device:0 Bus:2 Slot:0 Compute:1.2 Driver:373.6
02:30:07:OpenCL Device 1: Platform:0 Device:1 Bus:3 Slot:0 Compute:1.2 Driver:373.6
02:30:07:OpenCL Device 2: Platform:0 Device:2 Bus:4 Slot:0 Compute:1.2 Driver:373.6
02:30:07:OpenCL Device 3: Platform:0 Device:3 Bus:1 Slot:0 Compute:1.2 Driver:373.6
02:30:07:  Win32 Service: false
02:30:07:***********************************************************************
02:30:07:<config>
02:30:07:  <!-- Folding Core -->
02:30:07:  <checkpoint v='30'/>
02:30:07:  <core-priority v='low'/>
02:30:07:
02:30:07:  <!-- Folding Slot Configuration -->
02:30:07:  <client-type v='bigadv'/>
02:30:07:
02:30:07:  <!-- HTTP Server -->
02:30:07:  <allow v='127.0.0.1,192.168.1.253'/>
02:30:07:
02:30:07:  <!-- Network -->
02:30:07:  <proxy v=':8080'/>
02:30:07:
02:30:07:  <!-- Remote Command Server -->
02:30:07:  <password v='*******'/>
02:30:07:
02:30:07:  <!-- Slot Control -->
02:30:07:  <pause-on-battery v='false'/>
02:30:07:  <power v='FULL'/>
02:30:07:
02:30:07:  <!-- User Information -->
02:30:07:  <passkey v='********************************'/>
02:30:07:  <team v='224497'/>
02:30:07:  <user v='Aurum_ALL_1KJd683KCGdt9eV9DNUXRDxs4ZgQCEdURJ'/>
02:30:07:
02:30:07:  <!-- Work Unit Control -->
02:30:07:  <next-unit-percentage v='100'/>
02:30:07:
02:30:07:  <!-- Folding Slots -->
02:30:07:  <slot id='0' type='CPU'>
02:30:07:    <cpus v='18'/>
02:30:07:  </slot>
02:30:07:  <slot id='1' type='GPU'/>
02:30:07:  <slot id='2' type='GPU'/>
02:30:07:  <slot id='3' type='GPU'/>
02:30:07:  <slot id='4' type='GPU'/>
02:30:07:</config>

00:12:06:WU02:FS02:Upload 8.67%
00:12:06:ERROR:WU02:FS02:Exception: Transfer failed
00:12:06:WU02:FS02:Sending unit results: id:02 state:SEND error:NO_ERROR project:11402 run:0 clone:8 gen:523 core:0x21 unit:0x000002db8ca304f255ed4e3f8ab35596
00:12:06:WU02:FS02:Uploading 10.81MiB to 140.163.4.242
00:12:06:WU02:FS02:Connecting to 140.163.4.242:8080
00:12:07:WU01:FS02:0x21:Completed 1800000 out of 5000000 steps (36%)
00:12:36:WU06:FS01:Upload 7.42%
00:12:53:WU07:FS04:0x21:Completed 720000 out of 2000000 steps (36%)
00:12:54:WU05:FS03:Upload 9.67%
00:13:03:WU00:FS01:0x21:Completed 2075000 out of 2500000 steps (83%)
00:13:03:WU08:FS03:0x21:Completed 850000 out of 5000000 steps (17%)
00:13:34:WU02:FS02:Upload 0.58%
00:13:36:WU04:FS04:Upload 34.87%
00:13:42:WU06:FS01:Upload 7.71%
00:14:01:WU00:FS01:0x21:Completed 2100000 out of 2500000 steps (84%)
00:14:13:WU02:FS02:Upload 1.16%
00:14:15:WU05:FS03:Upload 10.06%
00:14:17:WU06:FS01:Upload 7.99%
00:14:20:WU04:FS04:Upload 35.33%
00:14:37:WU02:FS02:Upload 1.73%
00:14:46:WU03:FS00:0xa4:Completed 2350000 out of 2500000 steps  (94%)
00:14:55:WU07:FS04:0x21:Completed 740000 out of 2000000 steps (37%)
00:15:00:WU00:FS01:0x21:Completed 2125000 out of 2500000 steps (85%)
00:15:01:WARNING:FS03:Size of positions 5177 does not match topology 5059
00:15:01:WU04:FS04:Upload 35.79%
00:15:17:WU04:FS04:Upload 36.25%
00:15:17:WU02:FS02:Upload 2.31%
00:15:33:WU05:FS03:Upload 10.45%
00:15:50:WU04:FS04:Upload 36.70%
00:15:51:WU01:FS02:0x21:Completed 1850000 out of 5000000 steps (37%)
00:15:57:WU00:FS01:0x21:Completed 2150000 out of 2500000 steps (86%)
00:15:59:WU05:FS03:Upload 10.83%
00:16:10:WARNING:FS04:Size of positions 18932 does not match topology 18848
00:16:20:WU04:FS04:Upload 37.16%
00:16:27:WU05:FS03:Upload 11.22%
00:16:43:WU04:FS04:Upload 37.62%
00:16:44:WU08:FS03:0x21:Completed 900000 out of 5000000 steps (18%)
00:16:55:WU00:FS01:0x21:Completed 2175000 out of 2500000 steps (87%)
00:16:58:WU04:FS04:Upload 38.08%
00:17:06:WU07:FS04:0x21:Completed 760000 out of 2000000 steps (38%)
00:17:13:WU04:FS04:Upload 38.54%
00:17:27:WU04:FS04:Upload 39.00%
00:17:48:WU04:FS04:Upload 39.46%
00:17:52:WARNING:FS02:Size of positions 5217 does not match topology 2740
00:17:53:WU00:FS01:0x21:Completed 2200000 out of 2500000 steps (88%)
00:17:54:WU06:FS01:Upload 8.28%
00:18:10:WU06:FS01:Upload 8.57%
00:18:20:WU03:FS00:0xa4:Completed 2375000 out of 2500000 steps  (95%)
00:18:41:WU06:FS01:Upload 8.85%
00:18:52:WU00:FS01:0x21:Completed 2225000 out of 2500000 steps (89%)
00:18:59:WU04:FS04:Upload 39.92%
00:19:09:WU07:FS04:0x21:Completed 780000 out of 2000000 steps (39%)
00:19:17:WU06:FS01:Upload 9.14%
00:19:27:WU04:FS04:Upload 40.37%
00:19:29:WU05:FS03:Upload 11.61%
00:19:39:WU01:FS02:0x21:Completed 1900000 out of 5000000 steps (38%)
00:19:49:WU00:FS01:0x21:Completed 2250000 out of 2500000 steps (90%)
00:19:59:WU05:FS03:Upload 11.99%
00:20:12:WU06:FS01:Upload 9.42%
00:20:22:WU08:FS03:0x21:Completed 950000 out of 5000000 steps (19%)
00:20:38:WU04:FS04:Upload 40.83%
00:20:46:WU00:FS01:0x21:Completed 2275000 out of 2500000 steps (91%)
00:21:02:WU04:FS04:Upload 41.29%
00:21:10:WU05:FS03:Upload 12.38%
00:21:12:WU07:FS04:0x21:Completed 800000 out of 2000000 steps (40%)
00:21:32:WU04:FS04:Upload 41.75%
00:21:42:WU06:FS01:Upload 9.71%
00:21:44:WU00:FS01:0x21:Completed 2300000 out of 2500000 steps (92%)
00:21:53:WU03:FS00:0xa4:Completed 2400000 out of 2500000 steps  (96%)
00:21:58:WU04:FS04:Upload 42.21%
00:22:20:WU04:FS04:Upload 42.67%
00:22:27:WU02:FS02:Upload 2.89%
00:22:27:WARNING:WU02:FS02:Exception: Failed to send results to work server: Transfer failed
00:22:27:WU02:FS02:Trying to send results to collection server
00:22:27:WU02:FS02:Uploading 10.81MiB to 128.252.203.2
00:22:27:WU02:FS02:Connecting to 128.252.203.2:8080
00:22:43:WU00:FS01:0x21:Completed 2325000 out of 2500000 steps (93%)
00:23:00:WU04:FS04:Upload 43.13%
00:23:14:WU07:FS04:0x21:Completed 820000 out of 2000000 steps (41%)
00:23:25:WU01:FS02:0x21:Completed 1950000 out of 5000000 steps (39%)
00:23:30:WU04:FS04:Upload 43.59%
00:23:40:WU00:FS01:0x21:Completed 2350000 out of 2500000 steps (94%)
00:24:00:WU04:FS04:Upload 44.04%
00:24:01:WU08:FS03:0x21:Completed 1000000 out of 5000000 steps (20%)
00:24:09:WARNING:FS03:Size of positions 5177 does not match topology 5059
00:24:37:WU04:FS04:Upload 44.50%
00:24:37:WU00:FS01:0x21:Completed 2375000 out of 2500000 steps (95%)
00:25:05:WU04:FS04:Upload 44.96%
00:25:17:WU07:FS04:0x21:Completed 840000 out of 2000000 steps (42%)
00:25:26:WU03:FS00:0xa4:Completed 2425000 out of 2500000 steps  (97%)
00:25:33:WU04:FS04:Upload 45.42%
00:25:35:WU00:FS01:0x21:Completed 2400000 out of 2500000 steps (96%)
00:25:57:WU04:FS04:Upload 45.88%
00:26:14:WU02:FS02:Upload 0.58%
00:26:16:WU06:FS01:Upload 9.99%
00:26:16:ERROR:WU06:FS01:Exception: Transfer failed
00:26:16:WU06:FS01:Sending unit results: id:06 state:SEND error:NO_ERROR project:10496 run:74 clone:51 gen:11 core:0x21 unit:0x000000108ca304f558896aa21fd6ebb3
00:26:16:WU06:FS01:Uploading 21.89MiB to 140.163.4.245
00:26:16:WU06:FS01:Connecting to 140.163.4.245:8080
00:26:34:WU00:FS01:0x21:Completed 2425000 out of 2500000 steps (97%)
00:26:34:WU05:FS03:Upload 12.77%
00:26:34:ERROR:WU05:FS03:Exception: Transfer failed
00:26:35:WU05:FS03:Sending unit results: id:05 state:SEND error:NO_ERROR project:11428 run:0 clone:171 gen:15 core:0x21 unit:0x0000000f8ca304f158c845cfc0d748f2
00:26:35:WU05:FS03:Uploading 16.15MiB to 140.163.4.241
00:26:35:WU05:FS03:Connecting to 140.163.4.241:8080
00:26:50:WU06:FS01:Upload 0.29%
00:27:10:WU01:FS02:0x21:Completed 2000000 out of 5000000 steps (40%)
00:27:13:WU04:FS04:Upload 46.34%
00:27:19:WARNING:FS02:Size of positions 5217 does not match topology 2740
00:27:20:WU07:FS04:0x21:Completed 860000 out of 2000000 steps (43%)
00:27:31:WU00:FS01:0x21:Completed 2450000 out of 2500000 steps (98%)
00:27:35:WU04:FS04:Upload 46.80%
00:27:42:WU08:FS03:0x21:Completed 1050000 out of 5000000 steps (21%)
00:28:14:WU04:FS04:Upload 47.26%
00:28:28:WU00:FS01:0x21:Completed 2475000 out of 2500000 steps (99%)
00:28:59:WU04:FS04:Upload 47.71%
00:28:59:WU03:FS00:0xa4:Completed 2450000 out of 2500000 steps  (98%)
00:29:05:WARNING:FS04:Size of positions 18932 does not match topology 18848
00:29:24:WU04:FS04:Upload 48.17%
00:29:26:WU00:FS01:0x21:Completed 2500000 out of 2500000 steps (100%)
00:29:28:WU00:FS01:0x21:Saving result file logfile_01.txt
00:29:28:WU00:FS01:0x21:Saving result file checkpointState.xml
00:29:29:WU09:FS01:Connecting to 171.67.108.45:80
00:29:29:WU00:FS01:0x21:Saving result file checkpt.crc
00:29:29:WU00:FS01:0x21:Saving result file log.txt
00:29:29:WU00:FS01:0x21:Saving result file positions.xtc
00:29:30:WU00:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
00:29:31:WU07:FS04:0x21:Completed 880000 out of 2000000 steps (44%)
00:29:31:WU00:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
00:29:31:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9195 run:2 clone:75 gen:369 core:0x21 unit:0x000001f4ab40415c57cb2f11fe07644a
00:29:31:WU00:FS01:Uploading 5.72MiB to 171.64.65.92
00:29:31:WU00:FS01:Connecting to 171.64.65.92:8080
00:29:31:WU09:FS01:Assigned to work server 140.163.4.241
00:29:31:WU09:FS01:Requesting new work unit for slot 01: READY gpu:0:GP104 [GeForce GTX 1070] from 140.163.4.241
00:29:31:WU09:FS01:Connecting to 140.163.4.241:8080
00:29:34:WU09:FS01:Downloading 5.93MiB
00:29:39:WU06:FS01:Upload 0.57%
00:29:39:WARNING:WU06:FS01:Exception: Failed to send results to work server: Transfer failed
00:29:39:WU06:FS01:Trying to send results to collection server
00:29:39:WU06:FS01:Uploading 21.89MiB to 128.252.203.2
00:29:39:WU06:FS01:Connecting to 128.252.203.2:8080
00:29:40:WU09:FS01:Download 6.32%
00:29:46:WU09:FS01:Download 12.64%
00:29:49:WU02:FS02:Upload 1.16%
00:29:52:WU09:FS01:Download 22.13%
00:29:59:WU09:FS01:Download 38.99%
00:30:06:WU09:FS01:Download 49.53%
00:30:12:WU09:FS01:Download 56.90%
00:30:18:WU09:FS01:Download 63.22%
00:30:24:WU09:FS01:Download 75.87%
00:30:32:WU09:FS01:Download 87.46%
00:30:38:WU09:FS01:Download 96.94%
00:30:40:WU09:FS01:Download complete
00:30:40:WU09:FS01:Received Unit: id:09 state:DOWNLOAD error:NO_ERROR project:11428 run:1 clone:111 gen:9 core:0x21 unit:0x0000000a8ca304f158c8460838ec5a21
00:30:40:WU09:FS01:Starting
00:30:40:WU09:FS01:Running FahCore: "C:\Program Files\FAHClient/FAHCoreWrapper.exe" C:\Users\Rig-9\AppData\Roaming\FAHClient\cores/fahwebx.stanford.edu/cores/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 09 -suffix 01 -version 704 -lifeline 2956 -checkpoint 30 -opencl-platform 0 -gpu-vendor nvidia -gpu 0
00:30:40:WU09:FS01:Started FahCore on PID 4596
00:30:40:WU09:FS01:Core PID:2908
00:30:40:WU09:FS01:FahCore 0x21 started
00:30:41:WARNING:WU03:FS00:Detected clock skew (1 mins 10 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:30:41:WARNING:WU01:FS02:Detected clock skew (1 mins 10 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:30:41:WARNING:WU07:FS04:Detected clock skew (1 mins 11 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:30:41:WARNING:WU08:FS03:Detected clock skew (1 mins 11 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:30:41:WU09:FS01:0x21:*********************** Log Sta
rted 2017-04-12T00:30:41Z ***********************
00:30:41:WU09:FS01:0x21:Project: 11428 (Run 1, Clone 111, Gen 9)
00:30:41:WU09:FS01:0x21:Unit: 0x0000000a8ca304f158c8460838ec5a21
00:30:41:WU09:FS01:0x21:CPU: 0x00000000000000000000000000000000
00:30:41:WU09:FS01:0x21:Machine: 1
00:30:41:WU09:FS01:0x21:Reading tar file core.xml
00:30:41:WU09:FS01:0x21:Reading tar file system.xml
00:30:42:WU09:FS01:0x21:Reading tar file integrator.xml
00:30:42:WU09:FS01:0x21:Reading tar file state.xml
00:30:43:WU09:FS01:0x21:Digital signatures verified
00:30:43:WU09:FS01:0x21:Folding@home GPU Core21 Folding@home Core
00:30:43:WU09:FS01:0x21:Version 0.0.18
00:30:50:WU04:FS04:Upload 48.63%
00:30:57:WU09:FS01:0x21:Completed 0 out of 5000000 steps (0%)
00:30:57:WU09:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
00:30:59:WU01:FS02:0x21:Completed 2050000 out of 5000000 steps (41%)
00:31:14:WU04:FS04:Upload 49.09%
00:31:20:WU08:FS03:0x21:Completed 1100000 out of 5000000 steps (22%)
00:31:33:WU07:FS04:0x21:Completed 900000 out of 2000000 steps (45%)
00:31:44:WU04:FS04:Upload 49.55%
00:32:06:WU04:FS04:Upload 50.01%
00:32:31:WU03:FS00:0xa4:Completed 2475000 out of 2500000 steps  (99%)
00:32:40:WU04:FS04:Upload 50.47%
00:33:07:WU04:FS04:Upload 50.93%
00:33:16:WU00:FS01:Upload 1.09%
00:33:16:WARNING:WU00:FS01:Exception: Failed to send results to work server: Transfer failed
00:33:16:WU00:FS01:Trying to send results to collection server
00:33:16:WU00:FS01:Uploading 5.72MiB to 171.67.108.46
00:33:16:WU00:FS01:Connecting to 171.67.108.46:8080
00:33:17:WARNING:FS03:Size of positions 5177 does not match topology 5059
00:33:36:WU07:FS04:0x21:Completed 920000 out of 2000000 steps (46%)
00:33:36:WU05:FS03:Upload 0.39%
00:33:36:WARNING:WU05:FS03:Exception: Failed to send results to work server: Transfer failed
00:33:36:WU05:FS03:Trying to send results to collection server
00:33:36:WU05:FS03:Uploading 16.15MiB to 128.252.203.2
00:33:36:WU05:FS03:Connecting to 128.252.203.2:8080
00:33:52:WU00:FS01:Upload 1.09%
00:34:34:WU09:FS01:0x21:Completed 50000 out of 5000000 steps (1%)
00:34:44:WU01:FS02:0x21:Completed 2100000 out of 5000000 steps (42%)
00:34:51:WU05:FS03:Upload 0.39%
00:34:59:WU08:FS03:0x21:Completed 1150000 out of 5000000 steps (23%)
00:35:35:WU04:FS04:Upload 51.39%
00:35:38:WU07:FS04:0x21:Completed 940000 out of 2000000 steps (47%)
00:35:46:WU02:FS02:Upload 1.73%
00:35:46:WU00:FS01:Upload 2.19%
00:35:46:ERROR:WU00:FS01:Exception: Transfer failed
00:35:46:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9195 run:2 clone:75 gen:369 core:0x21 unit:0x000001f4ab40415c57cb2f11fe07644a
00:35:46:WU00:FS01:Uploading 5.72MiB to 171.64.65.92
00:35:46:WU00:FS01:Connecting to 171.64.65.92:8080
00:36:03:WU03:FS00:0xa4:Completed 2500000 out of 2500000 steps  (100%)
00:36:04:WU03:FS00:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
00:36:05:WU10:FS00:Connecting to 171.67.108.45:8080
00:36:06:WU10:FS00:Assigned to work server 155.247.166.220
00:36:06:WU10:FS00:Requesting new work unit for slot 00: RUNNING cpu:18 from 155.247.166.220
00:36:06:WU10:FS00:Connecting to 155.247.166.220:8080
00:36:10:WU10:FS00:Downloading 334.15KiB
00:36:14:WU03:FS00:0xa4:
00:36:14:WU03:FS00:0xa4:Finished Work Unit:
00:36:14:WU03:FS00:0xa4:- Reading up to 5216112 from "03/wudata_01.trr": Read 5216112
00:36:14:WU03:FS00:0xa4:trr file hash check passed.
00:36:14:WU03:FS00:0xa4:- Reading up to 515136 from "03/wudata_01.xtc": Read 515136
00:36:14:WU03:FS00:0xa4:xtc file hash check passed.
00:36:14:WU03:FS00:0xa4:edr file hash check passed.
00:36:14:WU03:FS00:0xa4:logfile size: 54851
00:36:14:WU03:FS00:0xa4:Leaving Run
00:36:15:WU10:FS00:Download complete
00:36:15:WU10:FS00:Received Unit: id:10 state:DOWNLOAD error:NO_ERROR project:8657 run:624 clone:0 gen:5 core:0xa4 unit:0x000000050002894c57f7f2a1a0419aab
00:36:17:WU03:FS00:0xa4:- Writing 5823803 bytes of core data to disk...
00:36:18:WU03:FS00:0xa4:Done: 5823291 -> 4941522 (compressed to 84.8 percent)
00:36:18:WU03:FS00:0xa4:  ... Done.
00:36:18:WU03:FS00:0xa4:- Shutting down core
00:36:18:WU03:FS00:0xa4:
00:36:18:WU03:FS00:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
00:36:19:WU03:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
00:36:19:WU03:FS00:Sending unit results: id:03 state:SEND error:NO_ERROR project:8623 run:0 clone:154 gen:23 core:0xa4 unit:0x000000190002894c57e2c92b68113a34
00:36:19:WU03:FS00:Uploading 4.71MiB to 155.247.166.220
00:36:19:WU03:FS00:Connecting to 155.247.166.220:8080
00:36:21:WU04:FS04:Upload 51.84%
00:37:39:WU07:FS04:0x21:Completed 960000 out of 2000000 steps (48%)
00:37:49:WU00:FS01:Upload 1.09%
00:38:08:WU06:FS01:Upload 0.29%
00:38:10:WU09:FS01:0x21:Completed 100000 out of 5000000 steps (2%)
00:38:31:WU01:FS02:0x21:Completed 2150000 out of 5000000 steps (43%)
00:38:36:WU08:FS03:0x21:Completed 1200000 out of 5000000 steps (24%)
00:38:43:WU00:FS01:Upload 2.19%
00:39:14:WU04:FS04:Upload 52.30%
00:39:14:WARNING:WU04:FS04:Exception: Failed to send results to work server: Transfer failed
00:39:14:WU04:FS04:Trying to send results to collection server
00:39:14:WU04:FS04:Uploading 13.62MiB to 171.67.108.46
00:39:14:WU04:FS04:Connecting to 171.67.108.46:8080
00:39:14:WU00:FS01:Upload 3.28%
00:39:38:WU07:FS04:0x21:Completed 980000 out of 2000000 steps (49%)
00:39:48:WU04:FS04:Upload 0.46%
00:39:59:WU02:FS02:Upload 2.31%
00:40:22:WU05:FS03:Upload 0.77%
00:40:44:WU04:FS04:Upload 0.92%
00:41:09:WU00:FS01:Upload 4.37%
00:41:22:WU04:FS04:Upload 1.38%
00:41:39:WU07:FS04:0x21:Completed 1000000 out of 2000000 steps (50%)
00:41:48:WU09:FS01:0x21:Completed 150000 out of 5000000 steps (3%)
00:41:49:WU04:FS04:Upload 1.84%
00:42:08:WU02:FS02:Upload 2.89%
00:42:08:ERROR:WU02:FS02:Exception: Transfer failed
00:42:08:WARNING:WU01:FS02:Size of positions 5217 does not match topology 2740
00:42:08:WARNING:WU01:FS02:Detected clock skew (5 mins 50 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:42:08:WARNING:WU07:FS04:Size of positions 18932 does not match topology 18848
00:42:08:WARNING:WU07:FS04:Detected clock skew (5 mins 50 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:42:08:WARNING:WU08:FS03:Detected clock skew (5 mins 50 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:42:08:WARNING:WU09:FS01:Size of positions 5206 does not match topology 2734
00:42:08:WARNING:WU09:FS01:Detected clock skew (5 mins 50 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:42:08:WU10:FS00:Starting
00:42:08:WU10:FS00:Running FahCore: "C:\Program Files\FAHClient/FAHCoreWrapper.exe" C:\Users\Rig-9\AppData\Roaming\FAHClient\cores/fahwebx.stanford.edu/cores/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 10 -suffix 01 -version 704 -lifeline 2956 -checkpoint 30 -np 18
00:42:08:WU10:FS00:Started FahCore on PID 868
00:42:08:WU10:FS00:Core PID:3736
00:42:08:WU10:FS00:FahCore 0xa4 started
00:42:09:WU02:FS02:Sending unit results: id:02 state:SEND error:NO_ERROR project:11402 run:0 clone:8 gen:523 core:0x21 unit:0x000002db8ca304f255ed4e3f8ab35596
00:42:09:WU02:FS02:Uploading 10.81MiB to 140.163.4.242
00:42:09:WU02:FS02:Connecting to 140.163.4.242:8080
00:42:09:WU10:FS00:0xa4:
00:42:09:WU10:FS00:0xa4:*------------------------------*
00:42:09:WU10:FS00:0xa4:Folding@Home Gromacs GB Core
00:42:09:WU10:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
00:42:09:WU10:FS00:0xa4:
00:42:09:WU10:FS00:0xa4:Preparing to commence simulation
00:42:09:WU10:FS00:0xa4:- Looking at optimizations...
00:42:09:WU10:FS00:0xa4:- Created dyn
00:42:09:WU10:FS00:0xa4:- Files status OK
00:42:09:WU10:FS00:0xa4:- Expanded 341655 -> 539432 (decompressed 157.8 percent)
00:42:09:WU10:FS00:0xa4:Called DecompressByteArray: compressed_data_size=341655 data_size=539432, decompressed_data_size=539432 diff=0
00:42:09:WU10:FS00:0xa4:- Digital signature verified
00:42:09:WU10:FS00:0xa4:
00:42:09:WU10:FS00:0xa4:Project: 8657 (Run 624, Clone 0, Gen 5)
00:42:09:WU10:FS00:0xa4:
00:42:09:WU10:FS00:0xa4:Assembly optimizations on if available.
00:42:09:WU10:FS00:0xa4:Entering M.D.
00:42:12:WU08:FS03:0x21:Completed 1250000 out of 5000000 steps (25%)
00:42:13:WU01:FS02:0x21:Completed 2200000 out of 5000000 steps (44%)
00:42:15:WU10:FS00:0xa4:Mapping NT from 18 to 18 
00:42:15:WU10:FS00:0xa4:Completed 0 out of 2500000 steps  (0%)
00:42:32:WU02:FS02:Upload 0.58%
00:42:57:WU00:FS01:Upload 5.46%
00:42:57:WARNING:WU00:FS01:Exception: Failed to send results to work server: Transfer failed
00:42:57:WU00:FS01:Trying to send results to collection server
00:42:57:WU00:FS01:Uploading 5.72MiB to 171.67.108.46
00:42:57:WU00:FS01:Connecting to 171.67.108.46:8080
00:43:17:WU02:FS02:Upload 1.16%
00:43:22:WU10:FS00:0xa4:Completed 25000 out of 2500000 steps  (1%)
00:43:28:WU03:FS00:Upload 1.33%
00:43:29:WU04:FS04:Upload 2.29%
00:43:29:ERROR:WU04:FS04:Exception: Transfer failed
00:43:29:WARNING:WU01:FS02:Detected clock skew (1 mins 21 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:43:29:WARNING:WU07:FS04:Detected clock skew (1 mins 21 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:43:29:WARNING:WU08:FS03:Size of positions 5177 does not match topology 5059
00:43:29:WARNING:WU08:FS03:Detected clock skew (1 mins 21 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:43:29:WARNING:WU09:FS01:Detected clock skew (1 mins 21 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:43:30:WU04:FS04:Sending unit results: id:04 state:SEND error:NO_ERROR project:9431 run:1392 clone:0 gen:5 core:0x21 unit:0x00000007ab436c9d586fdd3fead86fdf
00:43:30:WU04:FS04:Uploading 13.62MiB to 171.67.108.157
00:43:30:WU04:FS04:Connecting to 171.67.108.157:8080
00:43:46:WU07:FS04:0x21:Completed 1020000 out of 2000000 steps (51%)
00:43:52:WU04:FS04:Upload 0.46%
00:44:29:WU10:FS00:0xa4:Completed 50000 out of 2500000 steps  (2%)
00:44:31:WU04:FS04:Upload 0.92%
00:45:04:WU04:FS04:Upload 1.38%
00:45:25:WU09:FS01:0x21:Completed 200000 out of 5000000 steps (4%)
00:45:30:WU04:FS04:Upload 1.84%
00:45:37:WU10:FS00:0xa4:Completed 75000 out of 2500000 steps  (3%)
00:45:48:WU07:FS04:0x21:Completed 1040000 out of 2000000 steps (52%)
00:45:50:WU02:FS02:Upload 1.73%
00:45:50:WARNING:WU02:FS02:Exception: Failed to send results to work server: Transfer failed
00:45:50:WU02:FS02:Trying to send results to collection server
00:45:50:WU02:FS02:Uploading 10.81MiB to 128.252.203.2
00:45:50:WU02:FS02:Connecting to 128.252.203.2:8080
00:45:53:WU08:FS03:0x21:Completed 1300000 out of 5000000 steps (26%)
00:45:58:WU01:FS02:0x21:Completed 2250000 out of 5000000 steps (45%)
00:46:01:WU06:FS01:Upload 0.57%
00:46:19:WU05:FS03:Upload 1.16%
00:46:44:WU10:FS00:0xa4:Completed 100000 out of 2500000 steps  (4%)
00:46:46:WU04:FS04:Upload 2.29%
00:46:56:WU00:FS01:Upload 1.09%
00:47:50:WU10:FS00:0xa4:Completed 125000 out of 2500000 steps  (5%)
00:47:51:WU07:FS04:0x21:Completed 1060000 out of 2000000 steps (53%)
00:47:58:WU04:FS04:Upload 2.75%
00:48:13:WU05:FS03:Upload 1.55%
00:48:42:WU02:FS02:Upload 0.58%
00:48:57:WU10:FS00:0xa4:Completed 150000 out of 2500000 steps  (6%)
00:49:02:WU09:FS01:0x21:Completed 250000 out of 5000000 steps (5%)
00:49:31:WU08:FS03:0x21:Completed 1350000 out of 5000000 steps (27%)
00:49:32:WU02:FS02:Upload 1.16%
00:49:47:WU01:FS02:0x21:Completed 2300000 out of 5000000 steps (46%)
00:49:53:WU07:FS04:0x21:Completed 1080000 out of 2000000 steps (54%)
00:50:01:WU05:FS03:Upload 1.93%
00:50:04:WU10:FS00:0xa4:Completed 175000 out of 2500000 steps  (7%)
00:50:14:WU04:FS04:Upload 3.21%
00:50:27:WU00:FS01:Upload 2.19%
00:50:40:WU02:FS02:Upload 1.73%
00:51:11:WU10:FS00:0xa4:Completed 200000 out of 2500000 steps  (8%)
00:51:32:WU00:FS01:Upload 3.28%
00:51:55:WU07:FS04:0x21:Completed 1100000 out of 2000000 steps (55%)
00:52:17:WU04:FS04:Upload 3.67%
00:52:19:WU10:FS00:0xa4:Completed 225000 out of 2500000 steps  (9%)
00:52:44:WU09:FS01:0x21:Completed 300000 out of 5000000 steps (6%)
00:52:49:WU05:FS03:Upload 2.32%
00:53:12:WU08:FS03:0x21:Completed 1400000 out of 5000000 steps (28%)
00:53:25:WU10:FS00:0xa4:Completed 250000 out of 2500000 steps  (10%)
00:53:32:WU01:FS02:0x21:Completed 2350000 out of 5000000 steps (47%)
00:53:34:WU02:FS02:Upload 2.31%
00:53:58:WU07:FS04:0x21:Completed 1120000 out of 2000000 steps (56%)
00:53:58:WU05:FS03:Upload 2.71%
00:54:27:WU03:FS00:Upload 2.65%
00:54:31:WU05:FS03:Upload 3.10%
00:54:33:WU10:FS00:0xa4:Completed 275000 out of 2500000 steps  (11%)
00:54:56:WU06:FS01:Upload 0.86%
00:55:34:WU04:FS04:Upload 4.13%
00:55:40:WU10:FS00:0xa4:Completed 300000 out of 2500000 steps  (12%)
00:55:41:WU02:FS02:Upload 2.89%
00:55:41:ERROR:WU02:FS02:Exception: Transfer failed
00:55:41:WARNING:WU01:FS02:Size of positions 5217 does not match topology 2740
00:55:41:WARNING:WU01:FS02:Size of positions 5217 does not match topology 2740
00:55:41:WARNING:WU01:FS02:Detected clock skew (12 mins 12 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:55:41:WARNING:WU07:FS04:Size of positions 18932 does not match topology 18848
00:55:41:WARNING:WU07:FS04:Detected clock skew (12 mins 12 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:55:41:WARNING:WU08:FS03:Size of positions 5177 does not match topology 5059
00:55:41:WARNING:WU08:FS03:Detected clock skew (12 mins 12 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:55:41:WARNING:WU09:FS01:Size of positions 5206 does not match topology 2734
00:55:41:WARNING:WU09:FS01:Detected clock skew (12 mins 12 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:55:41:WARNING:WU10:FS00:Detected clock skew (12 mins 12 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:55:42:WU02:FS02:Sending unit results: id:02 state:SEND error:NO_ERROR project:11402 run:0 clone:8 gen:523 core:0x21 unit:0x000002db8ca304f255ed4e3f8ab35596
00:55:42:WU02:FS02:Uploading 10.81MiB to 140.163.4.242
00:55:42:WU02:FS02:Connecting to 140.163.4.242:8080
00:55:42:WU03:FS00:Upload 3.98%
00:56:08:WU07:FS04:0x21:Completed 1140000 out of 2000000 steps (57%)
00:56:13:WU04:FS04:Upload 4.59%
00:56:21:WU09:FS01:0x21:Completed 350000 out of 5000000 steps (7%)
00:56:46:WU04:FS04:Upload 5.05%
00:56:47:WU10:FS00:0xa4:Completed 325000 out of 2500000 steps  (13%)
00:56:50:WU08:FS03:0x21:Completed 1450000 out of 5000000 steps (29%)
00:56:58:WU02:FS02:Upload 0.58%
00:57:21:WU01:FS02:0x21:Completed 2400000 out of 5000000 steps (48%)
00:57:51:WU02:FS02:Upload 1.16%
00:57:51:WU04:FS04:Upload 5.51%
00:57:53:WU03:FS00:Upload 5.30%
00:57:53:WARNING:WU03:FS00:Exception: Failed to send results to work server: Transfer failed
00:57:53:WU03:FS00:Trying to send results to collection server
00:57:53:WU03:FS00:Uploading 4.71MiB to 155.247.166.219
00:57:53:WU03:FS00:Connecting to 155.247.166.219:8080
00:57:54:WU10:FS00:0xa4:Completed 350000 out of 2500000 steps  (14%)
00:58:11:WU07:FS04:0x21:Completed 1160000 out of 2000000 steps (58%)
00:58:20:WU04:FS04:Upload 5.96%
00:58:43:WU04:FS04:Upload 6.42%
00:59:01:WU10:FS00:0xa4:Completed 375000 out of 2500000 steps  (15%)
00:59:07:WU04:FS04:Upload 6.88%
00:59:28:WU05:FS03:Upload 3.48%
00:59:28:ERROR:WU05:FS03:Exception: Transfer failed
00:59:28:WARNING:WU01:FS02:Detected clock skew (3 mins 47 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:59:28:WARNING:WU07:FS04:Detected clock skew (3 mins 47 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:59:28:WARNING:WU08:FS03:Detected clock skew (3 mins 47 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:59:28:WARNING:WU09:FS01:Size of positions 5206 does not match topology 2734
00:59:28:WARNING:WU09:FS01:Detected clock skew (3 mins 47 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:59:28:WARNING:WU10:FS00:Detected clock skew (3 mins 47 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
00:59:28:WU05:FS03:Sending unit results: id:05 state:SEND error:NO_ERROR project:11428 run:0 clone:171 gen:15 core:0x21 unit:0x0000000f8ca304f158c845cfc0d748f2
00:59:28:WU05:FS03:Uploading 16.15MiB to 140.163.4.241
00:59:28:WU05:FS03:Connecting to 140.163.4.241:8080
00:59:39:WU04:FS04:Upload 7.34%
00:59:51:WU02:FS02:Upload 1.73%
00:59:51:WARNING:WU02:FS02:Exception: Failed to send results to work server: Transfer failed
00:59:51:WU02:FS02:Trying to send results to collection server
00:59:51:WU02:FS02:Uploading 10.81MiB to 128.252.203.2
00:59:51:WU02:FS02:Connecting to 128.252.203.2:8080
01:00:03:WU09:FS01:0x21:Completed 400000 out of 5000000 steps (8%)
01:00:07:WU10:FS00:0xa4:Completed 400000 out of 2500000 steps  (16%)
01:00:13:WU07:FS04:0x21:Completed 1180000 out of 2000000 steps (59%)
01:00:15:WU04:FS04:Upload 7.80%
01:00:28:WU08:FS03:0x21:Completed 1500000 out of 5000000 steps (30%)
01:00:47:WU04:FS04:Upload 8.26%
01:00:51:WU06:FS01:Upload 1.14%
01:00:54:WU00:FS01:Upload 4.37%
01:01:06:WU01:FS02:0x21:Completed 2450000 out of 5000000 steps (49%)
01:01:15:WU10:FS00:0xa4:Completed 425000 out of 2500000 steps  (17%)
01:01:21:WU00:FS01:Upload 5.46%
01:01:46:WU04:FS04:Upload 8.72%
01:02:15:WU07:FS04:0x21:Completed 1200000 out of 2000000 steps (60%)
01:02:17:WU04:FS04:Upload 9.18%
01:02:22:WU10:FS00:0xa4:Completed 450000 out of 2500000 steps  (18%)
01:02:27:WU00:FS01:Upload 6.56%
01:02:31:WU06:FS01:Upload 1.43%
01:02:31:ERROR:WU06:FS01:Exception: Transfer failed
01:02:31:WARNING:WU01:FS02:Detected clock skew (3 mins 03 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:02:31:WARNING:WU07:FS04:Detected clock skew (3 mins 03 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:02:31:WARNING:WU08:FS03:Size of positions 5177 does not match topology 5059
01:02:31:WARNING:WU08:FS03:Detected clock skew (3 mins 03 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:02:31:WARNING:WU09:FS01:Detected clock skew (3 mins 03 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:02:31:WARNING:WU10:FS00:Detected clock skew (3 mins 03 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:02:31:WU06:FS01:Sending unit results: id:06 state:SEND error:NO_ERROR project:10496 run:74 clone:51 gen:11 core:0x21 unit:0x000000108ca304f558896aa21fd6ebb3
01:02:31:WU06:FS01:Uploading 21.89MiB to 140.163.4.245
01:02:31:WU06:FS01:Connecting to 140.163.4.245:8080
01:03:01:WU04:FS04:Upload 9.63%
01:03:26:WU06:FS01:Upload 0.29%
01:03:28:WU10:FS00:0xa4:Completed 475000 out of 2500000 steps  (19%)
01:03:28:WU04:FS04:Upload 10.09%
01:03:40:WU09:FS01:0x21:Completed 450000 out of 5000000 steps (9%)
01:03:50:WU04:FS04:Upload 10.55%
01:04:10:WU08:FS03:0x21:Completed 1550000 out of 5000000 steps (31%)
01:04:18:WU07:FS04:0x21:Completed 1220000 out of 2000000 steps (61%)
01:04:23:WU00:FS01:Upload 7.65%
01:04:33:WU04:FS04:Upload 11.01%
01:04:35:WU10:FS00:0xa4:Completed 500000 out of 2500000 steps  (20%)
01:04:51:WU01:FS02:0x21:Completed 2500000 out of 5000000 steps (50%)
01:05:04:WU05:FS03:Upload 0.39%
01:05:04:WARNING:WU05:FS03:Exception: Failed to send results to work server: Transfer failed
01:05:04:WU05:FS03:Trying to send results to collection server
01:05:04:WU05:FS03:Uploading 16.15MiB to 128.252.203.2
01:05:04:WU05:FS03:Connecting to 128.252.203.2:8080
01:05:11:WU00:FS01:Upload 8.74%
01:05:38:WU00:FS01:Upload 9.83%
01:05:42:WU10:FS00:0xa4:Completed 525000 out of 2500000 steps  (21%)
01:05:51:WU02:FS02:Upload 0.58%
01:05:51:ERROR:WU02:FS02:Exception: Transfer failed
01:05:51:WARNING:WU01:FS02:Size of positions 5217 does not match topology 2740
01:05:51:WARNING:WU01:FS02:Detected clock skew (3 mins 20 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:05:51:WARNING:WU07:FS04:Detected clock skew (3 mins 20 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:05:51:WARNING:WU08:FS03:Detected clock skew (3 mins 20 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:05:51:WARNING:WU09:FS01:Detected clock skew (3 mins 20 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:05:51:WARNING:WU10:FS00:Detected clock skew (3 mins 20 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:05:51:WU02:FS02:Sending unit results: id:02 state:SEND error:NO_ERROR project:11402 run:0 clone:8 gen:523 core:0x21 unit:0x000002db8ca304f255ed4e3f8ab35596
01:05:51:WU02:FS02:Uploading 10.81MiB to 140.163.4.242
01:05:51:WU02:FS02:Connecting to 140.163.4.242:8080
01:06:09:WU00:FS01:Upload 10.93%
01:06:09:WU04:FS04:Upload 11.47%
01:06:15:WU00:FS01:Upload 12.02%
01:06:16:WU02:FS02:Upload 0.58%
01:06:20:WU07:FS04:0x21:Completed 1240000 out of 2000000 steps (62%)
01:06:23:WU02:FS02:Upload 1.16%
01:06:26:WU00:FS01:Upload 13.11%
01:06:29:WU04:FS04:Upload 11.93%
01:06:29:WU02:FS02:Upload 1.73%
01:06:36:WU04:FS04:Upload 12.39%
01:06:37:WU02:FS02:Upload 2.31%
01:06:40:WU00:FS01:Upload 14.20%
01:06:43:WU05:FS03:Upload 0.39%
01:06:44:WU02:FS02:Upload 2.89%
01:06:49:WU10:FS00:0xa4:Completed 550000 out of 2500000 steps  (22%)
01:06:50:WU00:FS01:Upload 15.30%
01:06:55:WU02:FS02:Upload 3.47%
01:06:56:WU05:FS03:Upload 0.77%
01:06:58:WU04:FS04:Upload 12.85%
01:07:01:WU00:FS01:Upload 16.39%
01:07:04:WU05:FS03:Upload 1.16%
01:07:11:WU02:FS02:Upload 4.05%
01:07:11:WU04:FS04:Upload 13.31%
01:07:13:WU05:FS03:Upload 1.55%
01:07:14:WU03:FS00:Upload 1.33%
01:07:18:WU09:FS01:0x21:Completed 500000 out of 5000000 steps (10%)
01:07:21:WU05:FS03:Upload 1.93%
01:07:25:WU00:FS01:Upload 17.48%
01:07:28:WU03:FS00:Upload 2.65%
01:07:31:WU02:FS02:Upload 4.62%
01:07:37:WU00:FS01:Upload 18.58%
01:07:39:WU04:FS04:Upload 13.76%
01:07:42:WU05:FS03:Upload 2.32%
01:07:45:WU02:FS02:Upload 5.20%
01:07:48:WU08:FS03:0x21:Completed 1600000 out of 5000000 steps (32%)
01:07:56:WU10:FS00:0xa4:Completed 575000 out of 2500000 steps  (23%)
01:07:57:WU04:FS04:Upload 14.22%
01:08:07:WU03:FS00:Upload 3.98%
01:08:08:WU04:FS04:Upload 14.68%
01:08:11:WU02:FS02:Upload 5.78%
01:08:24:WU03:FS00:Upload 5.30%
01:08:30:WU02:FS02:Upload 6.36%
01:08:32:WU07:FS04:0x21:Completed 1260000 out of 2000000 steps (63%)
01:08:36:WU00:FS01:Upload 19.67%
01:08:40:WU01:FS02:0x21:Completed 2550000 out of 5000000 steps (51%)
01:08:41:WU03:FS00:Upload 6.63%
01:08:45:WU02:FS02:Upload 6.94%
01:08:58:WU03:FS00:Upload 7.96%
01:09:02:WU10:FS00:0xa4:Completed 600000 out of 2500000 steps  (24%)
01:09:10:WU04:FS04:Upload 15.14%
01:09:21:WU00:FS01:Upload 20.76%
01:09:26:WU03:FS00:Upload 9.28%
01:09:27:WU06:FS01:Upload 0.57%
01:09:27:WARNING:WU06:FS01:Exception: Failed to send results to work server: Transfer failed
01:09:27:WU06:FS01:Trying to send results to collection server
01:09:27:WU06:FS01:Uploading 21.89MiB to 128.252.203.2
01:09:27:WU06:FS01:Connecting to 128.252.203.2:8080
01:09:28:WU02:FS02:Upload 7.51%
01:09:34:WU04:FS04:Upload 15.60%
01:09:54:WU03:FS00:Upload 10.61%
01:10:01:WU02:FS02:Upload 8.09%
01:10:09:WU10:FS00:0xa4:Completed 625000 out of 2500000 steps  (25%)
01:10:22:WU03:FS00:Upload 11.93%
01:10:24:WU02:FS02:Upload 8.67%
01:10:34:WU07:FS04:0x21:Completed 1280000 out of 2000000 steps (64%)
01:10:37:WU06:FS01:Upload 0.29%
01:10:43:WU03:FS00:Upload 13.26%
01:10:53:WU02:FS02:Upload 9.25%
01:10:55:WU04:FS04:Upload 16.06%
01:10:59:WU09:FS01:0x21:Completed 550000 out of 5000000 steps (11%)
01:11:13:WU02:FS02:Upload 9.83%
01:11:16:WU10:FS00:0xa4:Completed 650000 out of 2500000 steps  (26%)
01:11:28:WU08:FS03:0x21:Completed 1650000 out of 5000000 steps (33%)
01:11:32:WU03:FS00:Upload 14.59%
01:11:33:WU04:FS04:Upload 16.52%
01:11:42:WU02:FS02:Upload 10.40%
01:12:01:WU04:FS04:Upload 16.98%
01:12:15:WU04:FS04:Upload 17.43%
01:12:20:WU03:FS00:Upload 15.91%
01:12:23:WU10:FS00:0xa4:Completed 675000 out of 2500000 steps  (27%)
01:12:25:WU01:FS02:0x21:Completed 2600000 out of 5000000 steps (52%)
01:12:33:WU05:FS03:Upload 2.71%
01:12:36:WU07:FS04:0x21:Completed 1300000 out of 2000000 steps (65%)
01:12:42:WU02:FS02:Upload 10.98%
01:12:43:WU04:FS04:Upload 17.89%
01:13:03:WU04:FS04:Upload 18.35%
01:13:17:WU04:FS04:Upload 18.81%
01:13:30:WU10:FS00:0xa4:Completed 700000 out of 2500000 steps  (28%)
01:13:32:WU04:FS04:Upload 19.27%
01:13:33:WU02:FS02:Upload 11.56%
01:13:47:WU02:FS02:Upload 12.14%
01:14:06:WU03:FS00:Upload 17.24%
01:14:17:WU04:FS04:Upload 19.73%
01:14:23:WU02:FS02:Upload 12.72%
01:14:37:WU10:FS00:0xa4:Completed 725000 out of 2500000 steps  (29%)
01:14:37:WU09:FS01:0x21:Completed 600000 out of 5000000 steps (12%)
01:14:39:WU07:FS04:0x21:Completed 1320000 out of 2000000 steps (66%)
01:14:52:WU04:FS04:Upload 20.19%
01:15:01:WU03:FS00:Upload 18.57%
01:15:06:WU08:FS03:0x21:Completed 1700000 out of 5000000 steps (34%)
01:15:17:WU04:FS04:Upload 20.65%
01:15:33:WU03:FS00:Upload 19.89%
01:15:37:WU05:FS03:Upload 3.10%
01:15:43:WU10:FS00:0xa4:Completed 750000 out of 2500000 steps  (30%)
01:15:52:WU04:FS04:Upload 21.10%
01:16:09:WU03:FS00:Upload 21.22%
01:16:14:WU01:FS02:0x21:Completed 2650000 out of 5000000 steps (53%)
01:16:19:WU00:FS01:Upload 21.85%
01:16:23:WU05:FS03:Upload 3.48%
01:16:29:WU04:FS04:Upload 21.56%
01:16:41:WU07:FS04:0x21:Completed 1340000 out of 2000000 steps (67%)
01:16:44:WU02:FS02:Upload 13.29%
01:16:44:WARNING:WU02:FS02:Exception: Failed to send results to work server: Transfer failed
01:16:44:WU02:FS02:Trying to send results to collection server
01:16:44:WU02:FS02:Uploading 10.81MiB to 128.252.203.2
01:16:44:WU02:FS02:Connecting to 128.252.203.2:8080
01:16:44:WU05:FS03:Upload 3.87%
01:16:50:WU10:FS00:0xa4:Completed 775000 out of 2500000 steps  (31%)
01:16:59:WU04:FS04:Upload 22.02%
01:17:12:WU04:FS04:Upload 22.48%
01:17:23:WU04:FS04:Upload 22.94%
01:17:34:WU04:FS04:Upload 23.40%
01:17:45:WU04:FS04:Upload 23.86%
01:17:57:WU10:FS00:0xa4:Completed 800000 out of 2500000 steps  (32%)
01:17:58:WU04:FS04:Upload 24.32%
01:18:14:WU04:FS04:Upload 24.78%
01:18:18:WU09:FS01:0x21:Completed 650000 out of 5000000 steps (13%)
01:18:31:WU03:FS00:Upload 22.54%
01:18:41:WU02:FS02:Upload 0.58%
01:18:43:WU07:FS04:0x21:Completed 1360000 out of 2000000 steps (68%)
01:18:45:WU08:FS03:0x21:Completed 1750000 out of 5000000 steps (35%)
01:18:57:WU00:FS01:Upload 22.95%
01:19:04:WU10:FS00:0xa4:Completed 825000 out of 2500000 steps  (33%)
01:19:11:WU04:FS04:Upload 25.23%
01:19:13:WU00:FS01:Upload 24.04%
01:19:29:WU03:FS00:Upload 23.87%
01:19:46:WU04:FS04:Upload 25.69%
01:19:53:WU00:FS01:Upload 25.13%
01:19:59:WU01:FS02:0x21:Completed 2700000 out of 5000000 steps (54%)
01:20:10:WU00:FS01:Upload 26.22%
01:20:11:WU10:FS00:0xa4:Completed 850000 out of 2500000 steps  (34%)
01:20:24:WU00:FS01:Upload 27.32%
01:20:36:WU00:FS01:Upload 28.41%
01:20:40:WU06:FS01:Upload 0.57%
01:20:40:ERROR:WU06:FS01:Exception: Transfer failed
01:20:40:WARNING:WU01:FS02:Size of positions 5217 does not match topology 2740
01:20:40:WARNING:WU01:FS02:Detected clock skew (14 mins 49 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:20:41:WARNING:WU07:FS04:Size of positions 18932 does not match topology 18848
01:20:41:WARNING:WU07:FS04:Size of positions 18932 does not match topology 18848
01:20:41:WARNING:WU07:FS04:Detected clock skew (14 mins 50 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:20:41:WARNING:WU08:FS03:Size of positions 5177 does not match topology 5059
01:20:41:WARNING:WU08:FS03:Size of positions 5177 does not match topology 5059
01:20:41:WARNING:WU08:FS03:Detected clock skew (14 mins 50 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:20:41:WARNING:WU09:FS01:Size of positions 5206 does not match topology 2734
01:20:41:WARNING:WU09:FS01:Size of positions 5206 does not match topology 2734
01:20:41:WARNING:WU09:FS01:Detected clock skew (14 mins 50 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:20:41:WARNING:WU10:FS00:Detected clock skew (14 mins 50 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:20:41:WU06:FS01:Sending unit results: id:06 state:SEND error:NO_ERROR project:10496 run:74 clone:51 gen:11 core:0x21 unit:0x000000108ca304f558896aa21fd6ebb3
01:20:41:WU06:FS01:Uploading 21.89MiB to 140.163.4.245
01:20:41:WU06:FS01:Connecting to 140.163.4.245:8080
01:20:50:WU00:FS01:Upload 29.50%
01:20:53:WU05:FS03:Upload 4.26%
01:20:54:WU07:FS04:0x21:Completed 1380000 out of 2000000 steps (69%)
01:21:13:WU00:FS01:Upload 30.59%
01:21:18:WU10:FS00:0xa4:Completed 875000 out of 2500000 steps  (35%)
01:21:31:WU05:FS03:Upload 4.64%
01:21:32:WU06:FS01:Upload 0.29%
01:21:33:WU02:FS02:Upload 1.16%
01:21:33:ERROR:WU02:FS02:Exception: Transfer failed
01:21:34:WU02:FS02:Sending unit results: id:02 state:SEND error:NO_ERROR project:11402 run:0 clone:8 gen:523 core:0x21 unit:0x000002db8ca304f255ed4e3f8ab35596
01:21:34:WU02:FS02:Uploading 10.81MiB to 140.163.4.242
01:21:34:WU02:FS02:Connecting to 140.163.4.242:8080
01:21:46:WU00:FS01:Upload 31.69%
01:21:56:WU09:FS01:0x21:Completed 700000 out of 5000000 steps (14%)
01:22:05:WU06:FS01:Upload 0.57%
01:22:08:WU00:FS01:Upload 32.78%
01:22:25:WU10:FS00:0xa4:Completed 900000 out of 2500000 steps  (36%)
01:22:26:WU08:FS03:0x21:Completed 1800000 out of 5000000 steps (36%)
01:22:26:WU00:FS01:Upload 33.87%
01:22:37:WU00:FS01:Upload 34.96%
01:22:47:WU04:FS04:Upload 26.15%
01:22:56:WU07:FS04:0x21:Completed 1400000 out of 2000000 steps (70%)
01:23:02:WU00:FS01:Upload 36.06%
01:23:05:WU05:FS03:Upload 5.03%
01:23:05:ERROR:WU05:FS03:Exception: Transfer failed
01:23:05:WARNING:WU01:FS02:Detected clock skew (1 mins 32 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:23:05:WARNING:WU07:FS04:Detected clock skew (1 mins 32 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:23:05:WARNING:WU08:FS03:Detected clock skew (1 mins 32 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:23:05:WARNING:WU09:FS01:Detected clock skew (1 mins 32 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:23:05:WARNING:WU10:FS00:Detected clock skew (1 mins 32 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
01:23:06:WU05:FS03:Sending unit results: id:05 state:SEND error:NO_ERROR project:11428 run:0 clone:171 gen:15 core:0x21 unit:0x0000000f8ca304f158c845cfc0d748f2
01:23:06:WU05:FS03:Uploading 16.15MiB to 140.163.4.241
01:23:06:WU05:FS03:Connecting to 140.163.4.241:8080
01:23:21:WU00:FS01:Upload 37.15%
01:23:22:WU02:FS02:Upload 0.58%
01:23:22:WARNING:WU02:FS02:Exception: Failed to send results to work server: Transfer failed
01:23:22:WU02:FS02:Trying to send results to collection server
01:23:22:WU02:FS02:Uploading 10.81MiB to 128.252.203.2
01:23:22:WU02:FS02:Connecting to 128.252.203.2:8080
01:23:22:WU06:FS01:Upload 0.86%
01:23:31:WU10:FS00:0xa4:Completed 925000 out of 2500000 steps  (37%)
01:23:44:WU01:FS02:0x21:Completed 2750000 out of 5000000 steps (55%)
01:23:57:WU04:FS04:Upload 26.61%
01:24:17:WU00:FS01:Upload 38.24%
01:24:22:WU04:FS04:Upload 27.07%
01:24:27:WU05:FS03:Upload 0.39%
01:24:39:WU10:FS00:0xa4:Completed 950000 out of 2500000 steps  (38%)
01:24:39:WU03:FS00:Upload 25.20%
01:24:42:WU04:FS04:Upload 27.53%
01:24:44:WU06:FS01:Upload 1.14%
01:24:44:WARNING:WU06:FS01:Exception: Failed to send results to work server: Transfer failed
01:24:44:WU06:FS01:Trying to send results to collection server
01:24:44:WU06:FS01:Uploading 21.89MiB to 128.252.203.2
01:24:44:WU06:FS01:Connecting to 128.252.203.2:8080
01:24:59:WU07:FS04:0x21:Completed 1420000 out of 2000000 steps (71%)
01:25:09:WU03:FS00:Upload 26.52%
01:25:21:WU04:FS04:Upload 27.99%
01:25:33:WU09:FS01:0x21:Completed 750000 out of 5000000 steps (15%)
01:25:41:WU05:FS03:Upload 0.77%
01:25:41:WARNING:WU05:FS03:Exception: Failed to send results to work server: Transfer failed
01:25:41:WU05:FS03:Trying to send results to collection server
01:25:41:WU05:FS03:Uploading 16.15MiB to 128.252.203.2
01:25:41:WU05:FS03:Connecting to 128.252.203.2:8080
01:25:45:WU10:FS00:0xa4:Completed 975000 out of 2500000 steps  (39%)
01:25:53:WU02:FS02:Upload 0.58%
01:26:04:WU08:FS03:0x21:Completed 1850000 out of 5000000 steps (37%)
01:26:10:WU00:FS01:Upload 39.34%
01:26:22:WU04:FS04:Upload 28.45%
01:26:24:WU05:FS03:Upload 0.39%
01:26:46:WU03:FS00:Upload 27.85%
01:26:46:WU06:FS01:Upload 0.29%
01:26:52:WU10:FS00:0xa4:Completed 1000000 out of 2500000 steps  (40%)
01:26:56:WU00:FS01:Upload 40.43%
01:27:01:WU07:FS04:0x21:Completed 1440000 out of 2000000 steps (72%)
In Science We Trust Image
Aurum
Posts: 296
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: Collection Server 128.252.203.2 failures

Post by Aurum »

Code: Select all

20:42:12:WU01:FS01:0x21:Completed 6250000 out of 6250000 steps (100%)
20:42:13:WU01:FS01:0x21:Saving result file logfile_01.txt
20:42:13:WU01:FS01:0x21:Saving result file checkpointState.xml
20:42:13:WU01:FS01:0x21:Saving result file checkpt.crc
20:42:13:WU01:FS01:0x21:Saving result file log.txt
20:42:13:WU01:FS01:0x21:Saving result file positions.xtc
20:42:13:WU01:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
20:42:16:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
20:42:16:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1856 clone:0 gen:5 core:0x21 unit:0x00000007ab436c9d585e06da23fcdc39
20:42:16:WU01:FS01:Uploading 7.76MiB to 171.67.108.157
20:42:16:WU01:FS01:Connecting to 171.67.108.157:8080
20:44:59:WU01:FS01:Upload 0.81%
20:46:44:WU01:FS01:Upload 1.61%
20:53:11:WU01:FS01:Upload 2.42%
20:58:22:WU01:FS01:Upload 3.22%
20:58:22:WARNING:WU01:FS01:Exception: Failed to send results to work server: Transfer failed
20:58:22:WU01:FS01:Trying to send results to collection server
20:58:22:WU01:FS01:Uploading 7.76MiB to 171.67.108.46
20:58:22:WU01:FS01:Connecting to 171.67.108.46:8080
21:02:26:WU01:FS01:Upload 0.81%
21:12:54:WU01:FS01:Upload 1.61%
21:20:19:WU01:FS01:Upload 2.42%
21:23:49:WU01:FS01:Upload 3.22%
21:29:26:WU01:FS01:Upload 4.03%
21:35:04:WU01:FS01:Upload 4.84%
21:35:04:ERROR:WU01:FS01:Exception: Transfer failed
21:35:05:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1856 clone:0 gen:5 core:0x21 unit:0x00000007ab436c9d585e06da23fcdc39
21:35:05:WU01:FS01:Uploading 7.76MiB to 171.67.108.157
21:35:05:WU01:FS01:Connecting to 171.67.108.157:8080
21:35:13:WU01:FS01:Upload 0.81%
21:35:21:WU01:FS01:Upload 1.61%
21:35:27:WU01:FS01:Upload 2.42%
21:35:37:WU01:FS01:Upload 3.22%
21:35:51:WU01:FS01:Upload 4.03%
21:36:03:WU01:FS01:Upload 4.84%
21:36:17:WU01:FS01:Upload 5.64%
21:36:32:WU01:FS01:Upload 6.45%
21:36:48:WU01:FS01:Upload 7.25%
21:36:59:WU01:FS01:Upload 8.06%
21:37:22:WU01:FS01:Upload 8.86%
21:37:41:WU01:FS01:Upload 9.67%
21:38:16:WU01:FS01:Upload 10.48%
21:38:32:WU01:FS01:Upload 11.28%
21:38:46:WU01:FS01:Upload 12.09%
21:39:25:WU01:FS01:Upload 12.89%
21:39:47:WU01:FS01:Upload 13.70%
21:40:00:WU01:FS01:Upload 14.51%
21:40:28:WU01:FS01:Upload 15.31%
21:40:51:WU01:FS01:Upload 16.12%
21:41:21:WU01:FS01:Upload 16.92%
21:41:45:WU01:FS01:Upload 17.73%
21:42:04:WU01:FS01:Upload 18.53%
21:42:35:WU01:FS01:Upload 19.34%
21:45:25:WU01:FS01:Upload 20.15%
21:45:25:WARNING:WU01:FS01:Exception: Failed to send results to work server: Transfer failed
21:45:25:WU01:FS01:Trying to send results to collection server
21:45:25:WU01:FS01:Uploading 7.76MiB to 171.67.108.46
21:45:25:WU01:FS01:Connecting to 171.67.108.46:8080
21:46:08:WU01:FS01:Upload 0.81%
21:47:44:WU01:FS01:Upload 1.61%
21:48:17:WU01:FS01:Upload 2.42%
21:49:05:WU01:FS01:Upload 3.22%
21:49:46:WU01:FS01:Upload 4.03%
21:50:35:WU01:FS01:Upload 4.84%
21:51:40:WU01:FS01:Upload 5.64%
1:56:38:WU01:FS01:Upload 6.45%
22:11:36:WU01:FS01:Upload 7.25%
22:13:30:WU01:FS01:Upload 8.06%
22:22:56:WU01:FS01:Upload 8.86%
22:28:42:WU01:FS01:Upload 9.67%
22:30:20:WU01:FS01:Upload 10.48%
22:39:05:WU01:FS01:Upload 11.28%
22:41:46:WU01:FS01:Upload 12.09%
22:44:02:WU01:FS01:Upload 12.89%
22:45:11:WU01:FS01:Upload 13.70%
22:46:14:WU01:FS01:Upload 14.51%
22:49:30:WU01:FS01:Upload 15.31%
22:52:44:WU01:FS01:Upload 16.12%
22:52:44:ERROR:WU01:FS01:Exception: Transfer failed
22:52:44:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1856 clone:0 gen:5 core:0x21 unit:0x00000007ab436c9d585e06da23fcdc39
22:52:44:WU01:FS01:Uploading 7.76MiB to 171.67.108.157
22:52:44:WU01:FS01:Connecting to 171.67.108.157:8080
22:54:23:WU01:FS01:Upload 0.81%
22:59:52:WU01:FS01:Upload 1.61%
******************************* Date: 2017-04-11 *******************************
23:05:43:WU01:FS01:Upload 2.42%
23:06:48:WU01:FS01:Upload 3.22%
23:11:04:WU01:FS01:Upload 4.03%
23:11:45:WU01:FS01:Upload 4.84%
23:14:43:WU01:FS01:Upload 5.64%
23:17:49:WU01:FS01:Upload 6.45%
23:17:49:WARNING:WU01:FS01:Exception: Failed to send results to work server: Transfer failed
23:17:49:WU01:FS01:Trying to send results to collection server
23:17:49:WU01:FS01:Uploading 7.76MiB to 171.67.108.46
23:17:49:WU01:FS01:Connecting to 171.67.108.46:8080
23:18:47:WU01:FS01:Upload 0.81%
23:23:46:WU01:FS01:Upload 1.61%
23:23:47:ERROR:WU01:FS01:Exception: Transfer failed
23:23:47:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1856 clone:0 gen:5 core:0x21 unit:0x00000007ab436c9d585e06da23fcdc39
23:23:47:WU01:FS01:Uploading 7.76MiB to 171.67.108.157
23:23:47:WU01:FS01:Connecting to 171.67.108.157:8080
23:24:51:WU01:FS01:Upload 0.81%
23:26:39:WU01:FS01:Upload 1.61%
23:27:36:WU01:FS01:Upload 2.42%
23:27:46:WU01:FS01:Upload 4.84%
23:27:52:WU01:FS01:Upload 5.64%
23:28:00:WU01:FS01:Upload 7.25%
23:28:10:WU01:FS01:Upload 8.86%
23:28:20:WU01:FS01:Upload 9.67%
23:28:29:WU01:FS01:Upload 10.48%
23:28:39:WU01:FS01:Upload 11.28%
23:28:45:WU01:FS01:Upload 12.09%
23:28:52:WU01:FS01:Upload 12.89%
23:28:59:WU01:FS01:Upload 13.70%
23:29:08:WU01:FS01:Upload 14.51%
23:29:20:WU01:FS01:Upload 15.31%
23:29:31:WU01:FS01:Upload 16.12%
23:29:47:WU01:FS01:Upload 16.92%
23:30:10:WU01:FS01:Upload 17.73%
23:30:39:WU01:FS01:Upload 18.53%
23:33:08:WU01:FS01:Upload 19.34%
23:33:36:WU01:FS01:Upload 20.15%
23:35:47:WU01:FS01:Upload 20.95%
23:35:47:WARNING:WU01:FS01:Exception: Failed to send results to work server: Transfer failed
23:35:47:WU01:FS01:Trying to send results to collection server
23:35:47:WU01:FS01:Uploading 7.76MiB to 171.67.108.46
23:35:47:WU01:FS01:Connecting to 171.67.108.46:8080
23:40:18:WU01:FS01:Upload 0.81%
23:40:18:ERROR:WU01:FS01:Exception: Transfer failed
23:40:19:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1856 clone:0 gen:5 core:0x21 unit:0x00000007ab436c9d585e06da23fcdc39
23:40:19:WU01:FS01:Uploading 7.76MiB to 171.67.108.157
23:40:19:WU01:FS01:Connecting to 171.67.108.157:8080
23:42:22:WU01:FS01:Upload 0.81%
23:43:12:WU01:FS01:Upload 1.61%
23:47:18:WU01:FS01:Upload 2.42%
00:00:19:WU01:FS01:Upload 3.22%
00:11:13:WU01:FS01:Upload 4.03%
00:11:13:WARNING:WU01:FS01:Exception: Failed to send results to work server: Transfer failed
00:11:13:WU01:FS01:Trying to send results to collection server
00:11:13:WU01:FS01:Uploading 7.76MiB to 171.67.108.46
00:11:13:WU01:FS01:Connecting to 171.67.108.46:8080
00:13:05:WU01:FS01:Upload 0.81%
00:13:51:WU01:FS01:Upload 1.61%
00:16:58:WU01:FS01:Upload 2.42%
00:17:07:WU01:FS01:Upload 3.22%
00:17:21:WU01:FS01:Upload 4.03%
00:17:30:WU01:FS01:Upload 4.84%
00:19:57:WU01:FS01:Upload 5.64%
00:19:57:ERROR:WU01:FS01:Exception: Transfer failed
00:19:57:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1856 clone:0 gen:5 core:0x21 unit:0x00000007ab436c9d585e06da23fcdc39
00:19:57:WU01:FS01:Uploading 7.76MiB to 171.67.108.157
00:19:57:WU01:FS01:Connecting to 171.67.108.157:8080
00:20:34:WU01:FS01:Upload 0.81%
00:27:11:WU01:FS01:Upload 1.61%
00:27:11:WARNING:WU01:FS01:Exception: Failed to send results to work server: Transfer failed
00:27:11:WU01:FS01:Trying to send results to collection server
00:27:11:WU01:FS01:Uploading 7.76MiB to 171.67.108.46
00:27:11:WU01:FS01:Connecting to 171.67.108.46:8080
00:36:18:WU01:FS01:Upload 0.81%
00:36:18:ERROR:WU01:FS01:Exception: Transfer failed
00:36:18:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1856 clone:0 gen:5 core:0x21 unit:0x00000007ab436c9d585e06da23fcdc39
00:36:18:WU01:FS01:Uploading 7.76MiB to 171.67.108.157
00:36:18:WU01:FS01:Connecting to 171.67.108.157:8080
00:41:33:WU01:FS01:Upload 0.81%
00:43:32:WU01:FS01:Upload 1.61%
00:53:38:WU01:FS01:Upload 2.42%
00:56:20:WU01:FS01:Upload 3.22%
00:56:20:WARNING:WU01:FS01:Exception: Failed to send results to work server: Transfer failed
00:56:20:WU01:FS01:Trying to send results to collection server
00:56:20:WU01:FS01:Uploading 7.76MiB to 171.67.108.46
00:56:20:WU01:FS01:Connecting to 171.67.108.46:8080
01:03:08:WU01:FS01:Upload 0.81%
01:04:05:WU01:FS01:Upload 1.61%
01:09:20:WU01:FS01:Upload 2.42%
01:12:52:WU01:FS01:Upload 3.22%
01:14:00:WU01:FS01:Upload 4.03%
01:14:48:WU01:FS01:Upload 4.84%
01:15:50:WU01:FS01:Upload 5.64%
01:16:25:WU01:FS01:Upload 6.45%
01:16:58:WU01:FS01:Upload 7.25%
01:17:26:WU01:FS01:Upload 8.06%
01:17:46:WU01:FS01:Upload 8.86%
01:18:21:WU01:FS01:Upload 9.67%
01:22:48:WU01:FS01:Upload 10.48%
01:28:16:WU01:FS01:Upload 11.28%
01:29:34:WU01:FS01:Upload 12.09%
01:30:07:WU01:FS01:Upload 12.89%
01:35:24:WU01:FS01:Upload 13.70%
01:35:24:ERROR:WU01:FS01:Exception: Transfer failed
01:35:24:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1856 clone:0 gen:5 core:0x21 unit:0x00000007ab436c9d585e06da23fcdc39
01:35:24:WU01:FS01:Uploading 7.76MiB to 171.67.108.157
01:35:24:WU01:FS01:Connecting to 171.67.108.157:8080
In Science We Trust Image
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Collection Server 128.252.203.2 failures

Post by bruce »

How fast is your internet connection? (One good speed test is http://www.speedtest.net/ but you'll want to shutdown FAH and anything else that might be using your network while you run it.)

It's really difficult to estimate it when there are lots of results all trying to upload at the same time as FAH will try to send many at the same time which effectively shares fractions of the bandwidth but which is the wrong thing to do if your bandwidth is limited. Also, there's no way to tell if the problem is on your end or it's a problem with the servers.
Aurum
Posts: 296
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: Collection Server 128.252.203.2 failures

Post by Aurum »

1 up 6 down
In Science We Trust Image
Aurum
Posts: 296
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: Collection Server 128.252.203.2 failures

Post by Aurum »

It started working again and now all rigs have uploaded.
Please delete this thread.
In Science We Trust Image
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Collection Server 128.252.203.2 failures

Post by bruce »

Well, it was either that your internet connection was too slow or too unreliable or FAH was having to share bandwidth with too many other computers or too many applications that were hogging the bandwidth.

Since it's fine now, that eliminates the first of four. The next time it happens, monitor the line for errors (2 of 4) and figure out what else might be causing the problem (3 or 4 of 4)

During the last part of the second log, you were only able to upload 13.70% of 7.76MiB to 171.67.108.46 in ~39 minutes.
Post Reply