155.247.166.220 downloads stalled

Moderators: Site Moderators, FAHC Science Team

Re: 155.247.166.220 downloads stalled

Postby vvoelz » Thu May 07, 2020 6:12 pm

Thanks rickoic -- high traffic remains an issue, so we might put some assignment filters on to see if that helps.
User avatar
vvoelz
Pande Group Member
 
Posts: 485
Joined: Sun Dec 02, 2007 9:07 pm
Location: Temple University, Philadelphia PA

Re: 155.247.166.220 downloads stalled

Postby rickoic » Fri May 08, 2020 12:37 pm

Well got woke up early by thunderstorms moving through so checked and found only 1 gpu frozen on download at 3.65% and had sit there for 5 hours or so.
I'm folding because Dec 2005 I had radical prostate surgery.
Lost brother to spinal cancer, brother-in-law to prostate cancer.
Several 1st cousins lost and a few who have survived.
rickoic
 
Posts: 276
Joined: Sat May 23, 2009 5:49 pm
Location: Mississippi near Memphis, Tn

Re: 155.247.166.220 downloads stalled

Postby vvoelz » Fri May 08, 2020 12:41 pm

p.s. Our head software dev is "working on a rewrite of the WS networking code that should help a lot
with uploads/downloads." Fingers crossed.
User avatar
vvoelz
Pande Group Member
 
Posts: 485
Joined: Sun Dec 02, 2007 9:07 pm
Location: Temple University, Philadelphia PA

Re: 155.247.166.220 downloads stalled

Postby rickoic » Fri May 08, 2020 1:51 pm

Guru/witchdoctor/medicine man. Wish him heap big magic.
rickoic
 
Posts: 276
Joined: Sat May 23, 2009 5:49 pm
Location: Mississippi near Memphis, Tn

Re: 155.247.166.220 downloads stalled

Postby info2x » Sun May 10, 2020 5:08 pm

I wish them luck as well.

After several days of not being assigned to this server I got an assignment and failed to download again.
info2x
 
Posts: 17
Joined: Sun Apr 19, 2020 7:56 am

Re: 155.247.166.220 downloads stalled

Postby HaloJones » Mon May 11, 2020 3:28 am

just found one and had to reboot to clear
1x Titan X, 5x 1070, 1x 970, 1 x Ryzen 3600

Image
HaloJones
 
Posts: 861
Joined: Thu Jul 24, 2008 11:16 am

Re: 155.247.166.220 downloads stalled

Postby rickoic » Mon May 11, 2020 3:10 pm

Sat had 1 to reboot.
Sun had 2 to reboot
Mon had 2 to reboot
Plus through out the day Sun caught 3 more that I had to reboot.

Tks
Rick
rickoic
 
Posts: 276
Joined: Sat May 23, 2009 5:49 pm
Location: Mississippi near Memphis, Tn

Re: 155.247.166.220 downloads stalled

Postby rickoic » Mon May 18, 2020 6:14 pm

Appears Tech Guru had heap big magic. Been 3 days or more since I've had a problem with the sergver.

Tks for the good work.
rickoic
 
Posts: 276
Joined: Sat May 23, 2009 5:49 pm
Location: Mississippi near Memphis, Tn

Re: 155.247.166.220 downloads stalled

Postby cfhdev » Tue May 26, 2020 1:46 am

The last few days this has been stalling again.
cfhdev
 
Posts: 41
Joined: Tue Mar 17, 2020 2:10 am

Re: 155.247.166.220 downloads stalled

Postby PantherX » Tue May 26, 2020 8:48 am

I can access the landing page without issues. Thus, would chalk it up to peak traffic potentially saturating the bandwidth. If the situation worsens, please post the log file so we can investigate it further :)
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time

Welcome To The F@H Support Forum Ӂ Troubleshooting Bad WUs Ӂ Troubleshooting Server Connectivity Issues
User avatar
PantherX
Site Moderator
 
Posts: 6618
Joined: Wed Dec 23, 2009 10:33 am
Location: Land Of The Long White Cloud

Re: 155.247.166.220 downloads stalled

Postby cfhdev » Tue May 26, 2020 4:50 pm

I can to. I don't think the issue is just a bandwidth issue. The client should have a time out period where it will either dump the process and try again or switch to another server.

Example from log of WU02:FS02 from one client. Before and after the reboot. Normally I dump the WU from
Code: Select all
/var/lib/fahclient/work
this time just rebooted

Code: Select all
*********************** Log Started 2020-05-26T01:13:35Z ***********************
******************************* Date: 2020-05-26 *******************************
07:44:25:WU02:FS02:Connecting to assign1.foldingathome.org:80
07:44:25:WARNING:WU02:FS02:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
07:44:25:WU02:FS02:Connecting to assign2.foldingathome.org:80
07:44:26:WU02:FS02:Assigned to work server 155.247.166.220
07:44:26:WU02:FS02:Requesting new work unit for slot 02: RUNNING gpu:1:********* from 155.247.166.220
07:44:26:WU02:FS02:Connecting to 155.247.166.220:8080
07:44:26:WU02:FS02:Downloading 12.96MiB
******************************* Date: 2020-05-26 *******************************
******************************* Date: 2020-05-26 *******************************
*********************** Log Started 2020-05-26T15:22:32Z ***********************
*********************** Log Started 2020-05-26T15:22:32Z ***********************
15:23:19:WU02:FS02:Connecting to assign1.foldingathome.org:80
15:23:24:WARNING:WU02:FS02:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
15:23:24:WU02:FS02:Connecting to assign2.foldingathome.org:80
15:23:25:WU02:FS02:Assigned to work server 128.252.203.10
15:23:25:WU02:FS02:Requesting new work unit for slot 02: READY gpu:1:********* from 128.252.203.10
15:23:25:WU02:FS02:Connecting to 128.252.203.10:8080
15:25:34:WARNING:WU02:FS02:WorkServer connection failed on port 8080 trying 80
15:25:34:WU02:FS02:Connecting to 128.252.203.10:80
15:26:28:WU02:FS02:Downloading 86.23MiB
15:26:34:WU02:FS02:Download 20.58%
15:26:40:WU02:FS02:Download 36.38%
15:26:46:WU02:FS02:Download 45.23%
15:26:52:WU02:FS02:Download 52.69%
15:26:58:WU02:FS02:Download 61.32%
15:27:04:WU02:FS02:Download 68.71%
15:27:10:WU02:FS02:Download 77.84%
15:27:16:WU02:FS02:Download 89.22%
15:27:22:WU02:FS02:Download 97.77%
15:27:22:WU02:FS02:Download complete

It just stops/hangs in the middle of the download and never tries anything again until the reboot 8 hours later.
cfhdev
 
Posts: 41
Joined: Tue Mar 17, 2020 2:10 am

Re: 155.247.166.220 downloads stalled

Postby PantherX » Wed May 27, 2020 4:17 am

When you encounter the network issue, there's no need to delete any files. Simply restart FAHClient. It's a known bug: https://github.com/FoldingAtHome/fah-issues/issues/983
User avatar
PantherX
Site Moderator
 
Posts: 6618
Joined: Wed Dec 23, 2009 10:33 am
Location: Land Of The Long White Cloud

Re: 155.247.166.220 downloads stalled

Postby cfhdev » Sat May 30, 2020 3:39 am

Thanks for that bug page. Its interesting that it only happens for us on that one IP across multiple machines.
cfhdev
 
Posts: 41
Joined: Tue Mar 17, 2020 2:10 am

Re: 155.247.166.220 downloads stalled

Postby Sparkly » Wed Jun 10, 2020 8:56 am

This issue is still occurring a lot on this Server/IP – 155.247.166.220 by the looks of it and as far as I can see from my setups, which are currently running 12 slots total, it happens several times a day for me alone, thus needing a reboot of the process, since there is no recovery and the slot hangs in Ready.
Sparkly
 
Posts: 73
Joined: Sun Apr 19, 2020 12:01 pm

Re: 155.247.166.220 downloads stalled

Postby bruce » Wed Jun 10, 2020 5:27 pm

Yes, it happens on specific servers. They do get overloaded and data transfer rates become slower and slower ... and evenually it starts to hang which leads to the need to restart FAHClient. Posting topics like this one often leads to a correction on the server side. (except at night when we can't contact the appropriate people if they happen to have a life).

155.247.166.220 (vav4) reportedly had two CSs, one of which has failed. We reported a problem last night and it looks like somebody is working on it. (It was rebooted 44 minutes ago.)
bruce
 
Posts: 19861
Joined: Thu Nov 29, 2007 11:13 pm
Location: So. Cal.

PreviousNext

Return to Issues with a specific server

Who is online

Users browsing this forum: No registered users and 2 guests

cron