Can't connect to server

Moderators: Site Moderators, FAHC Science Team

Post Reply
MeeLee
Posts: 1375
Joined: Tue Feb 19, 2019 10:16 pm

Can't connect to server

Post by MeeLee »

Seems like my RTX2080Tis are exceptionally susceptible to this 'can't assign WU', or 'Can't connect to server' error.
Pausing/unpausing doesn't help.
The only way I can restart the WU, is by removing the slot, and re-entering it.

Anyone else going through the same?
JimboPalmer
Posts: 2573
Joined: Mon Feb 16, 2009 4:12 am
Location: Greenwood MS USA

Re: Can't connect to server

Post by JimboPalmer »

MeeLee you have watched who gets help and who doesn't enough to know the first 200 lines of your log will be vital to getting informed help.
Tsar of all the Rushers
I tried to remain childlike, all I achieved was childish.
A friend to those who want no friends
MeeLee
Posts: 1375
Joined: Tue Feb 19, 2019 10:16 pm

Re: Can't connect to server

Post by MeeLee »

I don't have my system here.
The log looks normal, save for the error line. I'll upload the part of the log that's necessary later, when I'm home.
I was just wondering if the servers were still being upgraded.
Had this issue for the past few weeks.
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Can't connect to server

Post by bruce »

Server upgrades have progressed slowly -- but it's unlikely to have caused your problem.

Unfortunately I don't have a way to tell the status of each of the projects that might be assigned to your system.

How many times did your Client retry and how long did it have to wait for an assignment? What was the last WU that was successfully assigned.
The only way I can restart the WU, is by removing the slot, and re-entering it.
That's strange. I don't understand why that would change anything.
MeeLee
Posts: 1375
Joined: Tue Feb 19, 2019 10:16 pm

Re: Can't connect to server

Post by MeeLee »

here's the snippet of the log.
At the end nothing happened. The GPU remained in 'ready' status, as you can see from the time code, for 4 hours, until I manually paused/unpaused it.
Only option is delete slot, and reinsert, not to lose valuable time of the other running GPUs

Code: Select all

00:28:06:WU02:FS01:Connecting to 65.254.110.245:8080
00:28:06:WU02:FS01:Assigned to work server 155.247.166.220
00:28:06:WU02:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:TU104 [GeForce RTX 2080] from 155.247.166.220
00:28:06:WU02:FS01:Connecting to 155.247.166.220:8080
00:28:07:WU02:FS01:Downloading 15.59MiB
00:28:13:WU02:FS01:Download 2.00%
00:28:20:WU02:FS01:Download 4.01%
00:28:27:WU02:FS01:Download 6.01%
00:28:31:WU06:FS01:0x21:Completed 500000 out of 500000 steps (100%)
00:28:33:WU02:FS01:Download 7.22%
00:28:35:WU06:FS01:0x21:Saving result file logfile_01.txt
00:28:35:WU06:FS01:0x21:Saving result file checkpointState.xml
00:28:35:WU06:FS01:0x21:Saving result file checkpt.crc
00:28:35:WU06:FS01:0x21:Saving result file log.txt
00:28:35:WU06:FS01:0x21:Saving result file positions.xtc
00:28:35:WU06:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
00:28:35:WU06:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
00:28:35:WU06:FS01:Sending unit results: id:06 state:SEND error:NO_ERROR project:14229 run:978 clone:0 gen:10 core:0x21 unit:0x0000000f80fccb0a5d6553dbcb2b0bd9
00:28:35:WU06:FS01:Uploading 51.14MiB to 128.252.203.10
00:28:35:WU06:FS01:Connecting to 128.252.203.10:8080
00:28:39:WU02:FS01:Download 9.22%
00:28:41:WU06:FS01:Upload 2.44%
00:28:47:WU06:FS01:Upload 7.45%
00:28:47:WU02:FS01:Download 10.82%
00:28:53:WU02:FS01:Download 13.23%
00:28:53:WU06:FS01:Upload 9.41%
00:28:59:WU06:FS01:Upload 11.85%
00:29:00:WU02:FS01:Download 14.83%
00:29:06:WU06:FS01:Upload 14.05%
00:29:09:WU02:FS01:Download 16.03%
00:29:13:WU06:FS01:Upload 16.25%
00:29:16:WU02:FS01:Download 17.24%
00:29:20:WU06:FS01:Upload 18.70%
00:29:23:WU02:FS01:Download 18.44%
00:29:29:WU02:FS01:Download 19.64%
00:29:31:WU06:FS01:Upload 20.90%
00:29:37:WU02:FS01:Download 20.44%
00:29:38:WU06:FS01:Upload 24.20%
00:29:44:WU02:FS01:Download 21.65%
00:29:51:WU02:FS01:Download 22.85%
00:29:51:WU06:FS01:Upload 28.72%
00:29:57:WU06:FS01:Upload 30.92%
00:29:59:WU02:FS01:Download 24.05%
00:30:04:WU06:FS01:Upload 33.12%
00:30:11:WU06:FS01:Upload 35.32%
00:30:18:WU06:FS01:Upload 37.52%
00:30:24:WU06:FS01:Upload 39.72%
00:30:30:WU06:FS01:Upload 41.92%
00:30:36:WU02:FS01:Download 24.45%
00:30:37:WU06:FS01:Upload 44.12%
00:30:44:WU06:FS01:Upload 46.32%
00:30:52:WU06:FS01:Upload 48.52%
00:31:03:WU06:FS01:Upload 52.79%
00:31:10:WU06:FS01:Upload 54.99%
00:31:18:WU02:FS01:Download 25.25%
00:31:19:WU06:FS01:Upload 57.19%
00:31:24:WU02:FS01:Download 26.46%
00:31:29:WU06:FS01:Upload 61.59%
00:31:30:WU02:FS01:Download 28.06%
00:31:37:WU06:FS01:Upload 63.79%
00:31:38:WU02:FS01:Download 28.86%
00:31:43:WU06:FS01:Upload 66.24%
00:31:44:WU02:FS01:Download 30.06%
00:31:51:WU06:FS01:Upload 68.44%
00:31:52:WU02:FS01:Download 31.67%
00:31:58:WU02:FS01:Download 32.87%
00:31:59:WU06:FS01:Upload 70.64%
00:32:07:WU02:FS01:Download 34.47%
00:32:11:WU06:FS01:Upload 75.03%
00:32:13:WU02:FS01:Download 35.67%
00:32:20:WU06:FS01:Upload 77.23%
00:32:24:WU02:FS01:Download 36.08%
00:32:26:WU06:FS01:Upload 79.43%
00:32:31:WU02:FS01:Download 37.28%
00:32:35:WU06:FS01:Upload 81.63%
00:32:39:WU02:FS01:Download 38.88%
00:32:45:WU06:FS01:Upload 83.96%
00:32:46:WU02:FS01:Download 40.48%
00:32:55:WU02:FS01:Download 41.29%
00:33:02:WU02:FS01:Download 42.09%
00:33:12:WU02:FS01:Download 42.89%
00:33:13:WU06:FS01:Upload 86.16%
00:33:19:WU02:FS01:Download 44.09%
00:33:26:WU02:FS01:Download 45.29%
00:33:32:WU06:FS01:Upload 88.48%
00:33:33:WU02:FS01:Download 46.50%
00:33:38:WU06:FS01:Upload 90.68%
00:33:41:WU02:FS01:Download 47.70%
00:33:45:WU06:FS01:Upload 92.88%
00:33:47:WU02:FS01:Download 48.50%
00:33:53:WU02:FS01:Download 48.90%
00:33:56:WU06:FS01:Upload 94.95%
00:34:19:WU06:FS01:Upload 97.52%
00:34:35:WU06:FS01:Upload 99.84%
00:35:01:WU06:FS01:Upload complete
00:35:01:WU06:FS01:Server responded WORK_ACK (400)
00:35:01:WU06:FS01:Final credit estimate, 49039.00 points
00:35:01:WU06:FS01:Cleaning up
******************************* Date: 2019-09-25 *****************************
04:40:46:FS01:Paused
04:40:48:FS01:Unpaused
04:41:14:FS01:Paused
04:41:17:FS01:Unpaused
MeeLee
Posts: 1375
Joined: Tue Feb 19, 2019 10:16 pm

Re: Can't connect to server

Post by MeeLee »

This is a copy of the error log (ignore the text before the 'Stalled',, that was still at setup phase)

Code: Select all

*********************** Log Started 2019-09-22T01:55:46Z ***********************
01:55:46:WARNING:WU03:Missing data files, dumping
01:55:47:WARNING:WU04:No longer matches Slot 2's configuration, migrating to FS01
01:55:47:ERROR:Exception: Unit not found
01:55:47:WARNING:WU05:No longer matches Slot 1's configuration, migrating to FS00
01:55:47:ERROR:Exception: Unit not found
01:55:49:ERROR:WU03:FS02:Exception: Server did not assign work unit
******************************* Date: 2019-09-23 *******************************
02:05:15:WARNING:WU00:FS03:Detected clock skew (1.01 days), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
02:05:15:WARNING:WU02:FS00:Detected clock skew (1.01 days), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
02:27:17:WARNING:WU04:FS03:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
02:31:35:WARNING:WU05:FS03:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
05:21:26:ERROR:WU03:FS01:Exception: Transfer failed
******************************* Date: 2019-09-23 *******************************
09:07:36:WARNING:WU04:FS02:FahCore returned: WU_STALLED (127 = 0x7f)
12:22:14:ERROR:WU00:FS03:Exception: Server did not assign work unit
12:58:26:ERROR:WU04:FS02:Exception: Server did not assign work unit
******************************* Date: 2019-09-23 *******************************
15:14:17:ERROR:WU03:FS02:Exception: Server did not assign work unit
18:57:43:ERROR:WU05:FS02:Exception: Server did not assign work unit
******************************* Date: 2019-09-23 *******************************
22:47:13:ERROR:WU00:FS01:Exception: Server did not assign work unit
22:54:07:WARNING:WU04:FS01:Exception: Failed to send results to work server: Transfer failed
23:03:51:ERROR:WU02:FS03:Exception: Transfer failed
00:11:57:ERROR:WU00:FS03:Exception: Transfer failed
00:25:48:ERROR:WU02:FS02:Exception: Transfer failed
******************************* Date: 2019-09-24 *******************************
07:50:57:ERROR:WU00:FS03:Exception: Server did not assign work unit
******************************* Date: 2019-09-24 *******************************
******************************* Date: 2019-09-24 *******************************
******************************* Date: 2019-09-24 *******************************
23:10:08:ERROR:WU06:FS01:Exception: Server did not assign work unit
23:12:23:ERROR:WU05:FS00:Exception: Transfer failed
23:16:42:WARNING:WU04:FS01:Exception: Failed to send results to work server: Transfer failed
23:18:09:WARNING:WU00:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
23:19:55:ERROR:WU05:FS00:Exception: Transfer failed
23:21:26:ERROR:WU05:FS00:Exception: Server did not assign work unit
23:21:29:ERROR:WU06:FS01:Exception: Transfer failed
23:24:12:ERROR:WU05:FS00:Exception: Transfer failed
23:24:20:ERROR:WU06:FS01:Exception: Transfer failed
23:27:57:ERROR:WU04:FS01:Exception: Transfer failed
23:28:21:ERROR:WU06:FS01:Exception: Transfer failed
23:33:26:ERROR:WU06:FS01:Exception: Transfer failed
23:33:34:ERROR:WU06:FS01:Exception: Server did not assign work unit
******************************* Date: 2019-09-25 *******************************
HaloJones
Posts: 920
Joined: Thu Jul 24, 2008 10:16 am

Re: Can't connect to server

Post by HaloJones »

I've been getting this too on multiple rigs, some Windows, some Linux. Seems like the download goes incredibly slowly, either stopping midway thru without error or finishing but the unit then not starting to fold. This is on systems that have done thousands of units (all gpu) without a problem. Seems to have become an issue over the last few weeks happening only from time to time and with no pattern I can discern
single 1070

Image
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Can't connect to server

Post by bruce »

I, too, had a similar problem last night. Apparently the servers at temple.edu had a problem which was (eventually) fixed but the communications connections also needed to be cleared out by a restart.
rwh202
Posts: 425
Joined: Mon Nov 15, 2010 8:51 pm
Hardware configuration: 8x GTX 1080
3x GTX 1080 Ti
3x GTX 1060
Various other bits and pieces
Location: South Coast, UK

Re: Can't connect to server

Post by rwh202 »

Still having problems with 155.247.166.220 even after restarts - diabolically slow downloads that eventually just hang. Tried restarting modem and router too, but no difference.
HaloJones
Posts: 920
Joined: Thu Jul 24, 2008 10:16 am

Re: Can't connect to server

Post by HaloJones »

Losing serious amounts of productive computing time due to this.
single 1070

Image
Catalina588
Posts: 41
Joined: Thu Oct 09, 2008 8:59 pm

Re: Can't connect to server

Post by Catalina588 »

This thread and the following thread are the same problem and symptoms. viewtopic.php?f=106&t=31878
Post Reply