54 failed attempts to upload to 66.170.111.50

Moderators: Site Moderators, FAHC Science Team

Re: 54 failed attempts to upload to 66.170.111.50

Postby Neil-B » Thu Oct 01, 2020 11:47 pm

Colonel_Klink wrote: ... and would think that I would have trouble with all servers if there was a problem on my end.


Not necessarily which is what I keep trying to say .. server location can cause routing issues so sometime a specific location for a server can cause issues from folders at other locations .. servers have different IP addresses - this can allow firewalls/proxies to have issues - iirc there have even been cases of av interference with comms when something related to wu trips heuristics - other times certain routing delays have caused connections to be dropped .. there are many different ways that a setup that works for most servers can have issues with others - that aren't as such the servers fault, simply more "environmental".

The fact that some WUs for that server get through is interesting and having PRCGs for ones that do as well as ones that don't might allow a pattern to be spotted.

That server is assigning some 3300 wus/hr and from reports so far the only person currently having issues appears to be yourself which as a data analyst (not linked in any way to fah other than as a folder) raises flags for me .. I have seen the deluge of "me to" posts when servers are having issues.

There will be something causing this issue - but it appears to be something very specific to the combination of your kit, your routing to that server, and that server ... from some of your logs it feels as if there might be a timeout/upload size pattern - which could be a server side issue - but if it is why aren't other folders reporting this?

I also seem to recall you may have had similar issues with other servers in the past (but I may well be wrong on this and haven't checked) ... again if these problems were many attempted returns to servers that otherwise appeared online and accepting wus (and not reported by other folders) these might have been linked to the current issue (even if different servers).

I honestly don't know for sure where the issue lies - if I did then it could be solved !! ... I am just very wary at saying it is an issue with the server that needs to be solved given the lack of any other reports - especially as if there isn't an issue with the server then it will never get resolved that way :(

The more data you can collect re WUs (prcg values, upload sizes, number of attempts before uploaded) that do upload and those that don't the more likely the cause can be tracked down and a solution found ... identifying if it is some form of time or size related pattern will also help

If you can manage to monitor the comms connections (I am not a networking specialist but I know some people are good at tracing routes and monitoring comms protocols) and see what is dropping the connection that might help (although that might be easier from server end depending on toolsets available).

I actually want this to be server side issue - and I so want to be wrong - as it will be possibly be easier to track down if it is ... but the silence of the masses makes my gut tell me otherwise :(
1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent, Quadro K420 1GB, FAH 7.6.13
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro, Quadro M1000M 2GB, FAH 7.6.13
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro, GTX 750Ti 2GB, FAH 7.6.13
Neil-B
 
Posts: 1410
Joined: Sun Mar 22, 2020 6:52 pm
Location: UK

Re: 54 failed attempts to upload to 66.170.111.50

Postby Neil-B » Thu Oct 01, 2020 11:49 pm

Great News :) ... Hopefully this wont happen again and it will just be one of those "unresolved mysteries" of the past
Neil-B
 
Posts: 1410
Joined: Sun Mar 22, 2020 6:52 pm
Location: UK

Re: 54 failed attempts to upload to 66.170.111.50

Postby bruce » Fri Oct 02, 2020 2:39 pm

My (unproven) theory was that this was an issue associated with the size of the upload packages. Assuming you do run into this again, next time compile a list of sizes (which is reported when the WU uploads) and see if "small" WUs get through to that server and "big" packes do not. (Sorry: I didn't take the time to do that from your posted logs, butt you're in a better position to do that anyway.) Each server might have its own demarcation of small vs. big. I did open a ticket for that server, so maybe that's what fixed it.
bruce
 
Posts: 20019
Joined: Thu Nov 29, 2007 11:13 pm
Location: So. Cal.

Re: 54 failed attempts to upload to 66.170.111.50

Postby Colonel_Klink » Fri Oct 02, 2020 6:17 pm

@Neil-B @bruce

Thanks for the last two messages! I feel better about the response to my issue.

I have an AMD 2900X set at 21 CPU's and have noticed over the last 2 weeks or so that there are times the CPU will sit idle for a few hours waiting with the "no WU's available for your configuration" message. If I have understood the plethora of messages I have read on the forum correctly, there are some projects that need large numbers of CPU's to process properly. If this is correct, does this mean that the resulting data file is larger than normal??? could this be part of the problem??? Also I have a RTX 2080 Super (I built this to play with the New version of Flight Simulator once this Covid mess starts to wind down next spring) and wonder if the GPU WU's that are assigned to me are also larger than normal. Also, the messages about no WU's available for my configuration started concurrently (well almost concurrently 3-4 later) than the first few issues I had with things not uploading in a timely manner. I also note that the problem does seem to go away after I mention it and someone responds that they have contacted the system manager responsible for the server.

I hate not understanding this system well enough to know when to get excited about stuff not working right. My F@H experience is limited to about 2 and half months of time on my Surface Laptop with a power scheme set to limit at 88% utilization to keep temperatures under control, during which time I never saw a single WU help up more than 3 attempts, which seems very reasonable to me. I've been running this Screaming Fast computer for about 7 weeks, and had no issues during the first couple of weeks when I was getting the fans tuned, to keep the temps reasonable. Based on some information I gathers on the Forum I chose to turn off the auto speed boost and have limited the processor speed to 3.76 MHZ which at full load gives me 68-70 C temps, which I am comfortable with.
Colonel_Klink
RTX 2080 Super
AMD Ryzen 9 3900X
Colonel_Klink
 
Posts: 41
Joined: Sat Aug 15, 2020 6:43 pm
Location: Pacific Northwest, USA

Re: 54 failed attempts to upload to 66.170.111.50

Postby Neil-B » Fri Oct 02, 2020 6:43 pm

There has been a bit of a scarcity of cpu WUs ... the headline figures on the server stats page initially look as if there are lots available but they include WUs on servers that aren't currently assigning ... whilst there are still WUs being assigned the high number of cpu folding slots out there means there is a bit of competition for them.

Some of this may be elated to the recent focus on moonshot gpu projects, the new cuda gpu core and trying to get a slew of new servers up and commissioned ... history tells me things will improve on this front
Neil-B
 
Posts: 1410
Joined: Sun Mar 22, 2020 6:52 pm
Location: UK

Re: 54 failed attempts to upload to 66.170.111.50

Postby bruce » Fri Oct 02, 2020 6:59 pm

The data being uploaded includes both the FAHlog and the science log, both of which can be compressed and which may grow in size if there are errors that were succesfully resumed from a checkpoint or whatever. It also includes a copy of the final WU's data which does depend on the number of atoms and isn't as easy to compress. I think it also includes snapshots for the Viewer and other miscellaneous items. Predicting the total size isn't really straightforward, which is probably why the project owner doesn't always adjust the server's maximum upload size. The actual size is reported in the message
10:17:07:WU01:FS00:Uploading 6.74MiB to 155.247.164.214
I don't know any way to see what the server's limit is set to.
bruce
 
Posts: 20019
Joined: Thu Nov 29, 2007 11:13 pm
Location: So. Cal.

Re: 54 failed attempts to upload to 66.170.111.50

Postby Colonel_Klink » Fri Oct 02, 2020 8:16 pm

@bruce,

How does 6.74MiB compare to other WU's being uploaded historically. Is this larger, smaller, or about the same?
Colonel_Klink
 
Posts: 41
Joined: Sat Aug 15, 2020 6:43 pm
Location: Pacific Northwest, USA

Re: 54 failed attempts to upload to 66.170.111.50

Postby Neil-B » Fri Oct 02, 2020 8:29 pm

Bruce will no doubt respond as/when he is on ... but big uploads to servers that have in the past caused issues the sizes have been 50+MiB (iirc some really bad ones were in the 100s of MiB) ... In one of the threads bruce actually states "Excessively large uploads are being rejected by the servers. (Yours is 68 MiB and it should be maybe 10 MiB)" ... In most of these cases multiple folders have reported issues about the servers ... but there are various things that can inflate size of uploads - and some of them can be specific to particular circumstances on specific folding kit ... Not sure limits on uploads but it would surprise me if these are normally less than 10MiB given previous posts on forum - but brice is betting of this so it would be brave of me to bet against him ;).
Neil-B
 
Posts: 1410
Joined: Sun Mar 22, 2020 6:52 pm
Location: UK

Re: 54 failed attempts to upload to 66.170.111.50

Postby PantherX » Sat Oct 03, 2020 10:24 am

By default, the Work Servers can accept 128 MB as upload. That can be changed if needed for Projects that are hosted on it. However, the client does perform compression if the Project supports it.

A WU that is failing to upload 50 times is an issue IMO. I generally upload majority of my WUs within the first attempt. If it takes longer than 10 attempts without a known issue on the Forum, then something is off. Since you mentioned laptop, are there any issues in Wi-Fi? If you have frequent packet drops due to poor Wi-Fi connectivity or interference, that could explain why you have to wait so many attempts to upload the WU.

If are comfortable/technical, you could install a VPN as a test and see if that has any impact on your upload issues. I assume that you have decent internet speed?
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time

Welcome To The F@H Support Forum Ӂ Troubleshooting Bad WUs Ӂ Troubleshooting Server Connectivity Issues
User avatar
PantherX
Site Moderator
 
Posts: 6765
Joined: Wed Dec 23, 2009 10:33 am
Location: Land Of The Long White Cloud

Re: 54 failed attempts to upload to 66.170.111.50

Postby bruce » Sat Oct 03, 2020 11:42 am

At the top of the previous page, I see a log ending with
Code: Select all
15:29:28:WARNING:WU02:FS00:Exception: Failed to send results to work server: Transfer failed
15:29:28:WU02:FS00:Sending unit results: id:02 state:SEND error:NO_ERROR

unit:0x0000002342aa6f325f618525f87ce26f
15:29:28:WU02:FS00:Uploading 9.92MiB to 66.170.111.50
15:29:28:WU02:FS00:Connecting to 66.170.111.50:8080
15:31:13:WU02:FS00:Upload 38.42%
15:31:13:WARNING:WU02:FS00:Exception: Failed to send results to work server: Transfer failed
15:31:13:WU02:FS00:Sending unit results: id:02 state:SEND error:NO_ERROR project:17407 run:0 clone:570 gen:34 core:0xa7 unit:0x0000002342aa6f325f618525f87ce26f
15:31:13:WU02:FS00:Uploading 9.92MiB to 66.170.111.50
15:31:13:WU02:FS00:Connecting to 66.170.111.50:8080


Clearly the file is just under 10 MiB isn't too big. Retrying everry 2 minutes isn't going to work and either it's a brand new upload or you're trying to force it, which isn't going to help. When did that WU finish processing?

According to https://apps.foldingathome.org/wu#proje ... 570&gen=34 you received credit for that WU early on 2020-09-30. You can't get credit for it more than once. The local copy should have been deleted when it was originally accepted.I just selected one example.


I just selected one example but it does not show a server error. Are all of your retries for WUs that have already been credited? As I said, that's not an error, that's a server error; that's a misinterpretation of what you're seeing. If that file is still retrying, ignore it and let it time-out or manually delete it.

Which ones are still retrying which also have not been credited? Did you restore a backup which created this problem? THe server SHOULD NOT be accepting attempts to re-upload the same files.
bruce
 
Posts: 20019
Joined: Thu Nov 29, 2007 11:13 pm
Location: So. Cal.

Re: 54 failed attempts to upload to 66.170.111.50

Postby bruce » Sat Oct 03, 2020 11:54 am

At the top of the previous page, I see a log ending with
Code: Select all
15:29:28:WARNING:WU02:FS00:Exception: Failed to send results to work server: Transfer failed
15:29:28:WU02:FS00:Sending unit results: id:02 state:SEND error:NO_ERROR

unit:0x0000002342aa6f325f618525f87ce26f
15:29:28:WU02:FS00:Uploading 9.92MiB to 66.170.111.50
15:29:28:WU02:FS00:Connecting to 66.170.111.50:8080
15:31:13:WU02:FS00:Upload 38.42%
15:31:13:WARNING:WU02:FS00:Exception: Failed to send results to work server: Transfer failed
15:31:13:WU02:FS00:Sending unit results: id:02 state:SEND error:NO_ERROR project:17407 run:0 clone:570 gen:34 core:0xa7 unit:0x0000002342aa6f325f618525f87ce26f
15:31:13:WU02:FS00:Uploading 9.92MiB to 66.170.111.50
15:31:13:WU02:FS00:Connecting to 66.170.111.50:8080


Clearly the file is just under 10 MiB isn't too big. It had just finished and was retrying every 2 minutes. According to https://apps.foldingathome.org/wu#proje ... 570&gen=34 you received credit for that WU early on 2020-09-30. You can't get credit for it more than once. If it's still re-trying, that's not a server error. It should have been deleted when it successfully uploaded. If it is still retrying, ignore it and let it time-out or manually delete it.

Which ones are still retrying which also have not been credited? Trying to upload over an unreliable connection is going to show retries but it should succeed and should delete the file.
bruce
 
Posts: 20019
Joined: Thu Nov 29, 2007 11:13 pm
Location: So. Cal.

Re: 54 failed attempts to upload to 66.170.111.50

Postby Neil-B » Sat Oct 03, 2020 12:55 pm

Sorry bruce, but as I read the thread and log at top of previous page was posted before the date credited !! ... and the OP has confirmed it did upload later in that page ... so the server wasn't accepting attempts to upload an already uploaded wu as you state.

I am really not convinced this is a server error ... since when has a server side issue only impacted a single folder ... repeatedly ... (and iirc on different servers previously) ... unless of course the server blacklists the IP address?
Neil-B
 
Posts: 1410
Joined: Sun Mar 22, 2020 6:52 pm
Location: UK

Re: 54 failed attempts to upload to 66.170.111.50

Postby bruce » Sat Oct 03, 2020 1:02 pm

OK, but we do need concrete examples of failures to work on. I selected something that wasn't an example of his problem. Where is the example of his actual problem?
bruce
 
Posts: 20019
Joined: Thu Nov 29, 2007 11:13 pm
Location: So. Cal.

Re: 54 failed attempts to upload to 66.170.111.50

Postby Colonel_Klink » Sat Oct 03, 2020 5:17 pm

@bruce @Neil-B

Yesterday afternoon all of the WU's that had completed had uploaded properly. This morning there are 4 for this same server that have tried to upload and failed between 11 and 25 times.
Thanks for showing me how to find i5 a WU has been credited
This WU has failed 25 times https://apps.foldingathome.org/wu#proje ... =46&gen=40 and no credit, oxa7
This WU has failed 24 times https://apps.foldingathome.org/wu#proje ... =32&gen=62 and no credit, 0x22
This WU has failed 11 times https://apps.foldingathome.org/wu#proje ... 971&gen=40 and no credit, 0xa7
This WU Has failed 23 times https://apps.foldingathome.org/wu#proje ... 918&gen=58 and no credit, 0xa7

Both CPU and GPU WU's have failed.

The code below shows 2 WU's in the same slot that have completed, one that initially failed and then uploaded and the second that has failed 25 times.

I do not know how to manually delete a WU and now that I know how to determine if I have received credit and for some reason the WU shows it is failing, I would like to know how to manually delete the WU.

Code: Select all
*********************** Log Started 2020-10-02T20:34:45Z ***********************
20:34:46:WU00:FS00:Connecting to assign1.foldingathome.org:80
20:34:47:WU00:FS00:Assigned to work server 155.247.164.214
20:34:47:WU00:FS00:Requesting new work unit for slot 00: READY cpu:21 from 155.247.164.214
20:34:47:WU00:FS00:Connecting to 155.247.164.214:8080
20:34:48:WU00:FS00:Downloading 2.84MiB
20:34:50:WU00:FS00:Download complete
20:34:50:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:14825 run:1639 clone:4 gen:13 core:0xa7 unit:0x0000000f9bf7a4d65ea48505f4871376
20:34:50:WU00:FS00:Starting
20:34:50:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\ejoep\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/win/64bit-avx-256/a7-0.0.19/Core_a7.fah/FahCore_a7.exe -dir 00 -suffix 01 -version 706 -lifeline 12368 -checkpoint 15 -np 21
20:34:50:WU00:FS00:Started FahCore on PID 1472
20:34:50:WU00:FS00:Core PID:1780
20:34:50:WU00:FS00:FahCore 0xa7 started
20:34:51:WU00:FS00:0xa7:*********************** Log Started 2020-10-02T20:34:50Z ***********************
20:34:51:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
20:34:51:WU00:FS00:0xa7:       Type: 0xa7
20:34:51:WU00:FS00:0xa7:       Core: Gromacs
20:34:51:WU00:FS00:0xa7:       Args: -dir 00 -suffix 01 -version 706 -lifeline 1472 -checkpoint 15 -np
20:34:51:WU00:FS00:0xa7:             21
20:34:51:WU00:FS00:0xa7:************************************ CBang *************************************
20:34:51:WU00:FS00:0xa7:       Date: Nov 27 2019
20:34:51:WU00:FS00:0xa7:       Time: 03:40:09
20:34:51:WU00:FS00:0xa7:   Revision: d25803215b59272441049dfa05a0a9bf7a6e3c48
20:34:51:WU00:FS00:0xa7:     Branch: master
20:34:51:WU00:FS00:0xa7:   Compiler: Visual C++ 2008
20:34:51:WU00:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
20:34:51:WU00:FS00:0xa7:   Platform: win32 10
20:34:51:WU00:FS00:0xa7:       Bits: 64
20:34:51:WU00:FS00:0xa7:       Mode: Release
20:34:51:WU00:FS00:0xa7:************************************ System ************************************
20:34:51:WU00:FS00:0xa7:        CPU: AMD Ryzen 9 3900X 12-Core Processor
20:34:51:WU00:FS00:0xa7:     CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
20:34:51:WU00:FS00:0xa7:       CPUs: 24
20:34:51:WU00:FS00:0xa7:     Memory: 31.93GiB
20:34:51:WU00:FS00:0xa7:Free Memory: 28.91GiB
20:34:51:WU00:FS00:0xa7:    Threads: WINDOWS_THREADS
20:34:51:WU00:FS00:0xa7: OS Version: 6.2
20:34:51:WU00:FS00:0xa7:Has Battery: false
20:34:51:WU00:FS00:0xa7: On Battery: false
20:34:51:WU00:FS00:0xa7: UTC Offset: -7
20:34:51:WU00:FS00:0xa7:        PID: 1780
20:34:51:WU00:FS00:0xa7:        CWD: C:\Users\ejoep\AppData\Roaming\FAHClient\work
20:34:51:WU00:FS00:0xa7:******************************** Build - libFAH ********************************
20:34:51:WU00:FS00:0xa7:    Version: 0.0.19
20:34:51:WU00:FS00:0xa7:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
20:34:51:WU00:FS00:0xa7:  Copyright: 2019 foldingathome.org
20:34:51:WU00:FS00:0xa7:   Homepage: https://foldingathome.org/
20:34:51:WU00:FS00:0xa7:       Date: Nov 25 2019
20:34:51:WU00:FS00:0xa7:       Time: 17:12:41
20:34:51:WU00:FS00:0xa7:   Revision: d5b5c747532224f986b7cd02c968ed9a20c16d6e
20:34:51:WU00:FS00:0xa7:     Branch: master
20:34:51:WU00:FS00:0xa7:   Compiler: Visual C++ 2008
20:34:51:WU00:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
20:34:51:WU00:FS00:0xa7:   Platform: win32 10
20:34:51:WU00:FS00:0xa7:       Bits: 64
20:34:51:WU00:FS00:0xa7:       Mode: Release
20:34:51:WU00:FS00:0xa7:************************************ Build *************************************
20:34:51:WU00:FS00:0xa7:       SIMD: avx_256
20:34:51:WU00:FS00:0xa7:********************************************************************************
20:34:51:WU00:FS00:0xa7:Project: 14825 (Run 1639, Clone 4, Gen 13)
20:34:51:WU00:FS00:0xa7:Unit: 0x0000000f9bf7a4d65ea48505f4871376
20:34:51:WU00:FS00:0xa7:Reading tar file core.xml
20:34:51:WU00:FS00:0xa7:Reading tar file frame13.tpr
20:34:51:WU00:FS00:0xa7:Digital signatures verified
20:34:51:WU00:FS00:0xa7:Calling: mdrun -s frame13.tpr -o frame13.trr -cpt 15 -nt 21
20:34:51:WU00:FS00:0xa7:Steps: first=0 total=250000
20:34:52:WU00:FS00:0xa7:Completed 1 out of 250000 steps (0%)
20:35:23:WU00:FS00:0xa7:Completed 2500 out of 250000 steps (1%)
20:35:53:WU00:FS00:0xa7:Completed 5000 out of 250000 steps (2%)

21:23:34:WU00:FS00:0xa7:Completed 242500 out of 250000 steps (97%)
21:24:04:WU00:FS00:0xa7:Completed 245000 out of 250000 steps (98%)
21:24:34:WU00:FS00:0xa7:Completed 247500 out of 250000 steps (99%)
21:25:04:WU00:FS00:0xa7:Completed 250000 out of 250000 steps (100%)
21:25:06:WU00:FS00:0xa7:Saving result file ..\logfile_01.txt
21:25:06:WU00:FS00:0xa7:Saving result file dhdl.xvg
21:25:06:WU00:FS00:0xa7:Saving result file frame13.trr
21:25:06:WU00:FS00:0xa7:Saving result file md.log
21:25:06:WU00:FS00:0xa7:Saving result file pullf.xvg
21:25:06:WU00:FS00:0xa7:Saving result file pullx.xvg
21:25:06:WU00:FS00:0xa7:Saving result file science.log
21:25:06:WU00:FS00:0xa7:Saving result file traj_comp.xtc
21:25:06:WU00:FS00:0xa7:Folding@home Core Shutdown: FINISHED_UNIT
21:25:06:WU00:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
21:25:06:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:14825 run:1639 clone:4 gen:13 core:0xa7 unit:0x0000000f9bf7a4d65ea48505f4871376
21:25:06:WU00:FS00:Uploading 6.81MiB to 155.247.164.214
21:25:06:WU00:FS00:Connecting to 155.247.164.214:8080
21:26:47:WU00:FS00:Upload 59.63%
21:26:47:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
21:26:47:WU00:FS00:Trying to send results to collection server
21:26:47:WU00:FS00:Uploading 6.81MiB to 155.247.166.220
21:26:47:WU00:FS00:Connecting to 155.247.166.220:8080
21:27:30:WU00:FS00:Upload 59.63%
21:28:31:WU00:FS00:Upload 60.55%
21:28:31:ERROR:WU00:FS00:Exception: Transfer failed
21:28:31:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:14825 run:1639 clone:4 gen:13 core:0xa7 unit:0x0000000f9bf7a4d65ea48505f4871376
21:28:31:WU00:FS00:Uploading 6.81MiB to 155.247.164.214
21:28:31:WU00:FS00:Connecting to 155.247.164.214:8080
21:30:14:WU00:FS00:Upload 59.63%
21:30:14:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
21:30:14:WU00:FS00:Trying to send results to collection server
21:30:14:WU00:FS00:Uploading 6.81MiB to 155.247.166.220
21:30:14:WU00:FS00:Connecting to 155.247.166.220:8080
21:32:09:WU00:FS00:Upload 59.63%
21:32:09:ERROR:WU00:FS00:Exception: Transfer failed
21:32:09:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:14825 run:1639 clone:4 gen:13 core:0xa7 unit:0x0000000f9bf7a4d65ea48505f4871376
21:32:10:WU00:FS00:Uploading 6.81MiB to 155.247.164.214
21:32:10:WU00:FS00:Connecting to 155.247.164.214:8080
21:32:16:WU00:FS00:Upload 55.04%
21:32:19:WU00:FS00:Upload complete
21:32:19:WU00:FS00:Server responded WORK_ACK (400)
21:32:19:WU00:FS00:Final credit estimate, 6953.00 points
21:32:19:WU00:FS00:Cleaning up
22:03:23:WU00:FS00:Connecting to assign1.foldingathome.org:80
22:03:23:WU00:FS00:Assigned to work server 66.170.111.50
22:03:23:WU00:FS00:Requesting new work unit for slot 00: RUNNING cpu:21 from 66.170.111.50
22:03:23:WU00:FS00:Connecting to 66.170.111.50:8080
22:03:24:WU00:FS00:Downloading 7.36MiB
22:03:25:WU00:FS00:Download complete
22:03:25:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
22:03:48:WU00:FS00:Starting
22:03:48:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\ejoep\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/win/64bit-avx-256/a7-0.0.19/Core_a7.fah/FahCore_a7.exe -dir 00 -suffix 01 -version 706 -lifeline 12368 -checkpoint 15 -np 21
22:03:48:WU00:FS00:Started FahCore on PID 7332
22:03:48:WU00:FS00:Core PID:6524
22:03:48:WU00:FS00:FahCore 0xa7 started
22:03:49:WU00:FS00:0xa7:*********************** Log Started 2020-10-02T22:03:48Z ***********************
22:03:49:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
22:03:49:WU00:FS00:0xa7:       Type: 0xa7
22:03:49:WU00:FS00:0xa7:       Core: Gromacs
22:03:49:WU00:FS00:0xa7:       Args: -dir 00 -suffix 01 -version 706 -lifeline 7332 -checkpoint 15 -np
22:03:49:WU00:FS00:0xa7:             21
22:03:49:WU00:FS00:0xa7:************************************ CBang *************************************
22:03:49:WU00:FS00:0xa7:       Date: Nov 27 2019
22:03:49:WU00:FS00:0xa7:       Time: 03:40:09
22:03:49:WU00:FS00:0xa7:   Revision: d25803215b59272441049dfa05a0a9bf7a6e3c48
22:03:49:WU00:FS00:0xa7:     Branch: master
22:03:49:WU00:FS00:0xa7:   Compiler: Visual C++ 2008
22:03:49:WU00:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
22:03:49:WU00:FS00:0xa7:   Platform: win32 10
22:03:49:WU00:FS00:0xa7:       Bits: 64
22:03:49:WU00:FS00:0xa7:       Mode: Release
22:03:49:WU00:FS00:0xa7:************************************ System ************************************
22:03:49:WU00:FS00:0xa7:        CPU: AMD Ryzen 9 3900X 12-Core Processor
22:03:49:WU00:FS00:0xa7:     CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
22:03:49:WU00:FS00:0xa7:       CPUs: 24
22:03:49:WU00:FS00:0xa7:     Memory: 31.93GiB
22:03:49:WU00:FS00:0xa7:Free Memory: 27.36GiB
22:03:49:WU00:FS00:0xa7:    Threads: WINDOWS_THREADS
22:03:49:WU00:FS00:0xa7: OS Version: 6.2
22:03:49:WU00:FS00:0xa7:Has Battery: false
22:03:49:WU00:FS00:0xa7: On Battery: false
22:03:49:WU00:FS00:0xa7: UTC Offset: -7
22:03:49:WU00:FS00:0xa7:        PID: 6524
22:03:49:WU00:FS00:0xa7:        CWD: C:\Users\ejoep\AppData\Roaming\FAHClient\work
22:03:49:WU00:FS00:0xa7:******************************** Build - libFAH ********************************
22:03:49:WU00:FS00:0xa7:    Version: 0.0.19
22:03:49:WU00:FS00:0xa7:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
22:03:49:WU00:FS00:0xa7:  Copyright: 2019 foldingathome.org
22:03:49:WU00:FS00:0xa7:   Homepage: https://foldingathome.org/
22:03:49:WU00:FS00:0xa7:       Date: Nov 25 2019
22:03:49:WU00:FS00:0xa7:       Time: 17:12:41
22:03:49:WU00:FS00:0xa7:   Revision: d5b5c747532224f986b7cd02c968ed9a20c16d6e
22:03:49:WU00:FS00:0xa7:     Branch: master
22:03:49:WU00:FS00:0xa7:   Compiler: Visual C++ 2008
22:03:49:WU00:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
22:03:49:WU00:FS00:0xa7:   Platform: win32 10
22:03:49:WU00:FS00:0xa7:       Bits: 64
22:03:49:WU00:FS00:0xa7:       Mode: Release
22:03:49:WU00:FS00:0xa7:************************************ Build *************************************
22:03:49:WU00:FS00:0xa7:       SIMD: avx_256
22:03:49:WU00:FS00:0xa7:********************************************************************************
22:03:49:WU00:FS00:0xa7:Project: 17408 (Run 0, Clone 46, Gen 40)
22:03:49:WU00:FS00:0xa7:Unit: 0x0000002d42aa6f325f618431680c3624
22:03:49:WU00:FS00:0xa7:Reading tar file core.xml
22:03:49:WU00:FS00:0xa7:Reading tar file frame40.tpr
22:03:49:WU00:FS00:0xa7:Digital signatures verified
22:03:49:WU00:FS00:0xa7:Calling: mdrun -s frame40.tpr -o frame40.trr -x frame40.xtc -cpt 15 -nt 21
22:03:49:WU00:FS00:0xa7:Steps: first=5000000 total=125000
22:03:51:WU00:FS00:0xa7:Completed 1 out of 125000 steps (0%)
22:04:19:WU00:FS00:0xa7:Completed 1250 out of 125000 steps (1%)
22:04:45:WU00:FS00:0xa7:Completed 2500 out of 125000 steps (2%)

22:46:35:WU00:FS00:0xa7:Completed 123750 out of 125000 steps (99%)
22:47:01:WU00:FS00:0xa7:Completed 125000 out of 125000 steps (100%)
22:47:05:WU00:FS00:0xa7:Saving result file ..\logfile_01.txt
22:47:05:WU00:FS00:0xa7:Saving result file frame40.trr
22:47:05:WU00:FS00:0xa7:Saving result file frame40.xtc
22:47:05:WU00:FS00:0xa7:Saving result file md.log
22:47:05:WU00:FS00:0xa7:Saving result file science.log
22:47:05:WU00:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
22:47:05:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
22:47:05:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
22:47:05:WU00:FS00:Connecting to 66.170.111.50:8080
22:48:45:WU00:FS00:Upload 35.30%
22:48:45:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
22:48:45:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
22:48:45:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
22:48:45:WU00:FS00:Connecting to 66.170.111.50:8080
22:50:37:WU00:FS00:Upload 35.93%
22:50:37:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
22:50:37:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
22:50:37:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
22:50:37:WU00:FS00:Connecting to 66.170.111.50:8080
22:52:30:WU00:FS00:Upload 35.30%
22:52:30:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
22:52:30:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
22:52:30:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
22:52:30:WU00:FS00:Connecting to 66.170.111.50:8080
22:52:38:WU00:FS00:Upload 35.93%
22:54:31:WU00:FS00:Upload 36.56%
22:54:31:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
22:55:07:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
22:55:07:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
22:55:07:WU00:FS00:Connecting to 66.170.111.50:8080
22:57:02:WU00:FS00:Upload 37.82%
22:57:02:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
22:59:22:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
22:59:22:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
22:59:22:WU00:FS00:Connecting to 66.170.111.50:8080
23:01:10:WU00:FS00:Upload 35.93%
23:01:10:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
23:06:13:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
23:06:13:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
23:06:13:WU00:FS00:Connecting to 66.170.111.50:8080
23:08:04:WU00:FS00:Upload 35.30%
23:08:04:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
23:17:19:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
23:17:19:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
23:17:19:WU00:FS00:Connecting to 66.170.111.50:8080
23:19:16:WU00:FS00:Upload 38.45%
23:19:16:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
23:35:15:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
23:35:15:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
23:35:15:WU00:FS00:Connecting to 66.170.111.50:8080
23:37:01:WU00:FS00:Upload 35.93%
23:37:01:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
00:04:18:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
00:04:18:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
00:04:18:WU00:FS00:Connecting to 66.170.111.50:8080
00:06:07:WU00:FS00:Upload 39.08%
00:06:07:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
00:51:17:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
00:51:17:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
00:51:17:WU00:FS00:Connecting to 66.170.111.50:8080
00:53:16:WU00:FS00:Upload 38.45%
00:53:16:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
01:51:17:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
01:51:17:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
01:51:17:WU00:FS00:Connecting to 66.170.111.50:8080
01:53:10:WU00:FS00:Upload 35.30%
01:53:10:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
******************************* Date: 2020-10-03 *******************************
02:51:17:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
02:51:17:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
02:51:17:WU00:FS00:Connecting to 66.170.111.50:8080
02:53:02:WU00:FS00:Upload 38.45%
02:53:03:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
03:51:17:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
03:51:17:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
03:51:17:WU00:FS00:Connecting to 66.170.111.50:8080
03:53:04:WU00:FS00:Upload 35.93%
03:53:04:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
04:51:17:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
04:51:17:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
04:51:17:WU00:FS00:Connecting to 66.170.111.50:8080
04:53:13:WU00:FS00:Upload 37.19%
04:53:13:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
05:51:17:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
05:51:17:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
05:51:17:WU00:FS00:Connecting to 66.170.111.50:8080
05:53:12:WU00:FS00:Upload 37.19%
05:53:13:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
06:51:17:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
06:51:17:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
06:51:17:WU00:FS00:Connecting to 66.170.111.50:8080
06:53:06:WU00:FS00:Upload 37.19%
06:53:06:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
07:51:18:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
07:51:18:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
07:51:18:WU00:FS00:Connecting to 66.170.111.50:8080
07:52:07:WU00:FS00:Upload 35.93%
07:53:07:WU00:FS00:Upload 36.56%
07:53:08:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
******************************* Date: 2020-10-03 *******************************
08:51:18:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
08:51:18:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
08:51:18:WU00:FS00:Connecting to 66.170.111.50:8080
08:51:27:WU00:FS00:Upload 38.45%
08:53:13:WU00:FS00:Upload 39.08%
08:53:13:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
09:51:18:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
09:51:18:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
09:51:18:WU00:FS00:Connecting to 66.170.111.50:8080
09:51:24:WU00:FS00:Upload 35.30%
09:53:10:WU00:FS00:Upload 35.93%
09:53:10:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
10:51:18:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
10:51:18:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
10:51:18:WU00:FS00:Connecting to 66.170.111.50:8080
10:51:30:WU00:FS00:Upload 35.30%
10:53:05:WU00:FS00:Upload 35.93%
10:53:05:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
11:51:18:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
11:51:18:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
11:51:18:WU00:FS00:Connecting to 66.170.111.50:8080
11:53:06:WU00:FS00:Upload 35.30%
11:53:06:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
12:51:18:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
12:51:18:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
12:51:18:WU00:FS00:Connecting to 66.170.111.50:8080
12:53:05:WU00:FS00:Upload 38.45%
12:53:05:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
13:51:18:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
13:51:19:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
13:51:19:WU00:FS00:Connecting to 66.170.111.50:8080
13:51:26:WU00:FS00:Upload 35.30%
13:53:13:WU00:FS00:Upload 35.93%
13:53:13:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
******************************* Date: 2020-10-03 *******************************
14:51:19:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
14:51:19:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
14:51:19:WU00:FS00:Connecting to 66.170.111.50:8080
14:51:33:WU00:FS00:Upload 35.30%
14:53:16:WU00:FS00:Upload 35.93%
14:53:16:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
15:51:19:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17408 run:0 clone:46 gen:40 core:0xa7 unit:0x0000002d42aa6f325f618431680c3624
15:51:19:WU00:FS00:Uploading 9.92MiB to 66.170.111.50
15:51:19:WU00:FS00:Connecting to 66.170.111.50:8080
15:53:02:WU00:FS00:Upload 36.56%
15:53:02:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
Colonel_Klink
 
Posts: 41
Joined: Sat Aug 15, 2020 6:43 pm
Location: Pacific Northwest, USA

Re: 54 failed attempts to upload to 66.170.111.50

Postby Colonel_Klink » Sat Oct 03, 2020 5:27 pm

FYI yesterday 2.5M PPD processed for me. So there are a lot of WU's processing fine.
Also, while I was preparing the previous post these two WU's also completed and started failing to upload to the same server.
https://apps.foldingathome.org/wu#proje ... =60&gen=42
https://apps.foldingathome.org/wu#proje ... 573&gen=48
Both CPU WU's
Colonel_Klink
 
Posts: 41
Joined: Sat Aug 15, 2020 6:43 pm
Location: Pacific Northwest, USA

PreviousNext

Return to Issues with a specific server

Who is online

Users browsing this forum: Yandex [Bot] and 2 guests

cron