Troubleshooting F@H Server Connectivity Issues (Please Read)

Moderators: Site Moderators, FAHC Science Team

Locked
PantherX
Site Moderator
Posts: 7020
Joined: Wed Dec 23, 2009 9:33 am
Hardware configuration: V7.6.21 -> Multi-purpose 24/7
Windows 10 64-bit
CPU:2/3/4/6 -> Intel i7-6700K
GPU:1 -> Nvidia GTX 1080 Ti
§
Retired:
2x Nvidia GTX 1070
Nvidia GTX 675M
Nvidia GTX 660 Ti
Nvidia GTX 650 SC
Nvidia GTX 260 896 MB SOC
Nvidia 9600GT 1 GB OC
Nvidia 9500M GS
Nvidia 8800GTS 320 MB

Intel Core i7-860
Intel Core i7-3840QM
Intel i3-3240
Intel Core 2 Duo E8200
Intel Core 2 Duo E6550
Intel Core 2 Duo T8300
Intel Pentium E5500
Intel Pentium E5400
Location: Land Of The Long White Cloud
Contact:

Troubleshooting F@H Server Connectivity Issues (Please Read)

Post by PantherX »

  • F@H Server Overview
A F@H Server can be classified into one of three categories:
  1. AS: Assignment Server
  2. WS: Work Server
  3. CS: Collection Server
For an operational status overview of F@H Servers, please visit this site: https://apps.foldingathome.org/serverstats


Server Status
The following are valid status and what they mean for the F@H Server:
  1. Assign: The Server will be assigning new WUs and receiving completed WUs.
  2. Accept: The Server will not be assigning new WUs but will only be receiving completed WUs.
  3. Down: The Server is not assigning new WUs nor is receiving completed WUs.
Below is a sample log where the client is communicating with the F@H Servers:

Code: Select all

07:14:33:WU01:FS01:Connecting to 65.254.110.245:8080
07:14:34:WU01:FS01:Assigned to work server 40.114.52.201
07:14:34:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 40.114.52.201
07:14:34:WU01:FS01:Connecting to 40.114.52.201:8080
07:14:55:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
07:14:55:WU01:FS01:Connecting to 40.114.52.201:80
07:15:15:WU01:FS01:Downloading 79.12MiB
07:15:21:WU01:FS01:Download 2.21%
Note: There is no need to change the logging level in V7 as it does not provide any additional information that would be useful in troubleshooting the issue. Instead, it will be a hindrance and slow down the troubleshooting process due to unnecessary lines of information
Last edited by PantherX on Wed Oct 13, 2010 4:19 am, edited 1 time in total.
PantherX
Site Moderator
Posts: 7020
Joined: Wed Dec 23, 2009 9:33 am
Hardware configuration: V7.6.21 -> Multi-purpose 24/7
Windows 10 64-bit
CPU:2/3/4/6 -> Intel i7-6700K
GPU:1 -> Nvidia GTX 1080 Ti
§
Retired:
2x Nvidia GTX 1070
Nvidia GTX 675M
Nvidia GTX 660 Ti
Nvidia GTX 650 SC
Nvidia GTX 260 896 MB SOC
Nvidia 9600GT 1 GB OC
Nvidia 9500M GS
Nvidia 8800GTS 320 MB

Intel Core i7-860
Intel Core i7-3840QM
Intel i3-3240
Intel Core 2 Duo E8200
Intel Core 2 Duo E6550
Intel Core 2 Duo T8300
Intel Pentium E5500
Intel Pentium E5400
Location: Land Of The Long White Cloud
Contact:

Troubleshooting The Assignment Server (AS)

Post by PantherX »

  • Assignment Server (AS)
Before any folding occurs, your F@H Client must be able to connect to the Assignment Server. A F@H Client which has been installed for the first time on a system, will request a User ID from the AS. Please note that the AS does not issue WUs. They keep track of the status of the Work Servers and assigns you to a WS that can give you an appropriate WU, depending on your F@H Clients configuration and available WUs. There are currently 2 AS Servers in production:
  1. assign1.foldingathome.org
  2. assign2.foldingathome.org
Troubleshooting Network Issues
If your F@H Client is unable to connect to the AS, it will not be assigned a WU thus would not be folding and will be idle instead.

Below is a sample log where the client has encountered a failure to connect to the AS:

Code: Select all

08:41:47:ERROR:WU01:FS01:Exception: Could not get IP address for assign1.foldingathome.org: No such host is known. 
08:41:47:ERROR:WU01:FS01:Exception: Could not get IP address for assign2.foldingathome.org: No such host is known. 
08:41:47:WARNING:WU01:FS01:Exception: Failed to find any IP addresses for assignment servers
08:41:47:ERROR:WU01:FS01:Exception: Could not get an assignment
08:41:47:ERROR:WU01:FS01:Exception: Could not get IP address for assign1.foldingathome.org: No such host is known. 
08:41:47:ERROR:WU01:FS01:Exception: Could not get IP address for assign2.foldingathome.org: No such host is known. 
08:41:47:WARNING:WU01:FS01:Exception: Failed to find any IP addresses for assignment servers
08:41:47:ERROR:WU01:FS01:Exception: Could not get an assignment
When this happens, most of the time, there is an issue is on the F@H Donor's side. The following should be checked:
  1. Internet Connection: Make sure that you are online.
  2. Router: Make sure that it configured correctly. F@H Clients use port 80 and 8080 only.
  3. Firewall (OS\Software\Hardware): Make sure that you have configured the Firewall to allow F@H Client to connect to the internet.
  4. Filtering: Ensure that there's no web filtering which is interfering with the connections to F@H Servers.
  5. Installed Software(s): Make sure that any internet monitoring software is configured to allow the F@H Client to connect to the internet. For example, Anti-Spyware software which may be included with your Firewall, Anti-Virus, or other Anti-Malware software.
Once you have checked the above, please try opening these sites in your browser from where the F@H Client is installed:
  1. F@H AS1: https://assign1.foldingathome.org/
  2. F@H AS2: https://assign2.foldingathome.org/
The correct and expected response should be a web page with the F@H logo and the text "Assignment Server" followed by a version number.
PantherX
Site Moderator
Posts: 7020
Joined: Wed Dec 23, 2009 9:33 am
Hardware configuration: V7.6.21 -> Multi-purpose 24/7
Windows 10 64-bit
CPU:2/3/4/6 -> Intel i7-6700K
GPU:1 -> Nvidia GTX 1080 Ti
§
Retired:
2x Nvidia GTX 1070
Nvidia GTX 675M
Nvidia GTX 660 Ti
Nvidia GTX 650 SC
Nvidia GTX 260 896 MB SOC
Nvidia 9600GT 1 GB OC
Nvidia 9500M GS
Nvidia 8800GTS 320 MB

Intel Core i7-860
Intel Core i7-3840QM
Intel i3-3240
Intel Core 2 Duo E8200
Intel Core 2 Duo E6550
Intel Core 2 Duo T8300
Intel Pentium E5500
Intel Pentium E5400
Location: Land Of The Long White Cloud
Contact:

Troubleshooting The Work Server (WS)

Post by PantherX »

  • Work Server (AS)
These F@H Servers hold new WUs and completed WUs. The researcher manages the WS to ensure that it is operational. There are multiple WS and you can view them here: https://apps.foldingathome.org/serverstats
WS plays an important role in:
  1. Assigning New WUs: It provides a new WU for the F@H Client to download and fold.
  2. Collecting Completed WU: It accepts the completed WU from the F@H Client, verifies it and once successful, assigned credit to it.
Downloading WU Issues
The AS should automatically be redirecting you to another WS shortly. If you're not being redirected, you will see these messages instead:

Code: Select all

18:06:52:WARNING:WU01:FS01:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration
This means that there are no WS available to assign a WU that matches your request. There's no need to report to report this as an issue.

Code: Select all

18:38:44:WU01:FS01:Assigned to work server 192.0.2.1
This means that your system configuration is not supported by F@H. If in doubt, create a new thread with your log file for verification.

It is recommended that you leave your client as is as it will try to get a new WU. The wait time between next attempts exponentially increases to prevent overloading the servers. You may pause/unpause the slot if the next attempt is in few hours. Alternatively, if don't mind receiving pre-release WUs, you can configure your client with <client-type v='advanced'/>


Uploading WU Issues
The client will automatically try to send the completed WU to the WS that assigned it. If the WS is not functioning as expected, the F@H Client is designed to deal with this by:
  1. If the initial attempts to upload the result to the WS fails, it might attempt to upload to the Collection Server (details below).
  2. If it still fails to upload the result, it will keep it in the work folder and will request a new WU to fold. The autosend feature will attempt to upload the result periodically.
Below is a sample log where the initial upload the WU fails but a subsequent attempt was successful:

Code: Select all

02:37:43:WU00:FS01:0x22:Completed 1000000 out of 1000000 steps (100%)
02:38:07:WU00:FS01:0x22:Saving result file ..\logfile_01.txt
02:38:07:WU00:FS01:0x22:Saving result file checkpointState.xml
02:38:07:WU00:FS01:0x22:Saving result file checkpt.crc
02:38:07:WU00:FS01:0x22:Saving result file positions.xtc
02:38:07:WU00:FS01:0x22:Saving result file science.log
02:38:08:WU00:FS01:0x22:Folding@home Core Shutdown: FINISHED_UNIT
02:38:08:WU00:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
02:38:08:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:13877 run:0 clone:1647 gen:10 core:0x22 unit:0x0000000d34e06d4a5e80cfea8a0b710b
02:38:08:WU00:FS01:Uploading 48.12MiB to 52.224.109.74
02:38:08:WU00:FS01:Connecting to 52.224.109.74:8080
02:38:52:WU00:FS01:Upload 0.26%
02:38:52:WARNING:WU00:FS01:Exception: Failed to send results to work server: Transfer failed
02:38:52:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:13877 run:0 clone:1647 gen:10 core:0x22 unit:0x0000000d34e06d4a5e80cfea8a0b710b
02:38:52:WU00:FS01:Uploading 48.12MiB to 52.224.109.74
02:38:52:WU00:FS01:Connecting to 52.224.109.74:8080
02:39:00:WU00:FS01:Upload 0.13%
02:39:43:WU00:FS01:Upload 0.26%
02:39:43:WARNING:WU00:FS01:Exception: Failed to send results to work server: Transfer failed
02:39:52:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:13877 run:0 clone:1647 gen:10 core:0x22 unit:0x0000000d34e06d4a5e80cfea8a0b710b
02:39:52:WU00:FS01:Uploading 48.12MiB to 52.224.109.74
02:39:52:WU00:FS01:Connecting to 52.224.109.74:8080
02:40:13:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
02:40:13:WU00:FS01:Connecting to 52.224.109.74:80
02:40:35:WARNING:WU00:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
02:41:30:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:13877 run:0 clone:1647 gen:10 core:0x22 unit:0x0000000d34e06d4a5e80cfea8a0b710b
02:41:30:WU00:FS01:Uploading 48.12MiB to 52.224.109.74
02:41:30:WU00:FS01:Connecting to 52.224.109.74:8080
02:41:51:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
02:41:51:WU00:FS01:Connecting to 52.224.109.74:80
02:42:12:WARNING:WU00:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
02:44:07:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:13877 run:0 clone:1647 gen:10 core:0x22 unit:0x0000000d34e06d4a5e80cfea8a0b710b
02:44:07:WU00:FS01:Uploading 48.12MiB to 52.224.109.74
02:44:07:WU00:FS01:Connecting to 52.224.109.74:8080
02:44:28:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
02:44:28:WU00:FS01:Connecting to 52.224.109.74:80
02:44:31:WU00:FS01:Upload 0.13%
02:44:48:WU00:FS01:Upload 0.26%
02:45:13:WU00:FS01:Upload 0.39%
...
02:52:33:WU00:FS01:Upload 98.57%
02:52:39:WU00:FS01:Upload 99.48%
02:52:43:WU00:FS01:Upload complete
02:52:43:WU00:FS01:Server responded WORK_ACK (400)
02:52:43:WU00:FS01:Final credit estimate, 184118.00 points
02:52:43:WU00:FS01:Cleaning up
Troubleshooting Network Issues
If you can connect to the WS via the web browser (using the IP address from the log or via the hostname via the Server Status page), the correct and expected response should be a web page with the F@H logo and the text "Work Server" followed by a version number. If you can't see that, then it is likely to be an issue with the F@H Server so please create a new topic in this forum with the IP address of the Server if one does not already exists: viewforum.php?f=18

Note: Credit for the WU is calculated from the time the WU was downloaded from the WS to when it was successfully uploaded to either the WS or the CS. Any delays in transmitting the completed WU will negatively impact your credit.
PantherX
Site Moderator
Posts: 7020
Joined: Wed Dec 23, 2009 9:33 am
Hardware configuration: V7.6.21 -> Multi-purpose 24/7
Windows 10 64-bit
CPU:2/3/4/6 -> Intel i7-6700K
GPU:1 -> Nvidia GTX 1080 Ti
§
Retired:
2x Nvidia GTX 1070
Nvidia GTX 675M
Nvidia GTX 660 Ti
Nvidia GTX 650 SC
Nvidia GTX 260 896 MB SOC
Nvidia 9600GT 1 GB OC
Nvidia 9500M GS
Nvidia 8800GTS 320 MB

Intel Core i7-860
Intel Core i7-3840QM
Intel i3-3240
Intel Core 2 Duo E8200
Intel Core 2 Duo E6550
Intel Core 2 Duo T8300
Intel Pentium E5500
Intel Pentium E5400
Location: Land Of The Long White Cloud
Contact:

Details Of Collection Server (CS)

Post by PantherX »

  • Collection Server (CS)
A Collection Server can only receive completed WUs. A single WS can have one or more CS assigned to it. The configuration of a CS is entirely optional and dependent on the Project and Work Server. If a CS is specified, the IP address will be displayed. If a CS is not specified, you will see 0.0.0.0 which is normal and not an issue.

Below is a sample log where the initial upload the WU to the WS fails so the client tries to upload to the CS and is successful:

Code: Select all

23:21:17:WU01:FS01:0x22:Completed 1000000 out of 1000000 steps (100%)
23:21:22:WU01:FS01:0x22:Saving result file ..\logfile_01.txt
23:21:22:WU01:FS01:0x22:Saving result file checkpointState.xml
23:21:22:WU01:FS01:0x22:Saving result file checkpt.crc
23:21:22:WU01:FS01:0x22:Saving result file positions.xtc
23:21:22:WU01:FS01:0x22:Saving result file science.log
23:21:22:WU01:FS01:0x22:Folding@home Core Shutdown: FINISHED_UNIT
23:21:23:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
23:21:23:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11776 run:0 clone:14194 gen:18 core:0x22 unit:0x00000021287234c95e74331e5ccbb06c
23:21:23:WU01:FS01:Uploading 50.04MiB to 40.114.52.201
23:21:23:WU01:FS01:Connecting to 40.114.52.201:8080
23:21:44:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:21:44:WU01:FS01:Connecting to 40.114.52.201:80
23:22:05:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 40.114.52.201:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
23:22:05:WU01:FS01:Trying to send results to collection server
23:22:05:WU01:FS01:Uploading 50.04MiB to 52.224.109.74
23:22:05:WU01:FS01:Connecting to 52.224.109.74:8080
23:22:38:WU01:FS01:Upload 0.37%
23:22:44:WU01:FS01:Upload 2.00%
23:30:38:WU01:FS01:Upload 98.66%
23:30:44:WU01:FS01:Upload 99.91%
23:30:48:WU01:FS01:Upload complete
23:30:49:WU01:FS01:Server responded WORK_ACK (400)
23:30:49:WU01:FS01:Final credit estimate, 119724.00 points
23:30:49:WU01:FS01:Cleaning up
Note: Credit for the WU is calculated from the time the WU was downloaded from the WS to when it was successfully uploaded to either the WS or the CS. Any delays in transmitting the completed WU will negatively impact your credit. If there are any delays in transmitting the completed WU from the CS to the WS, it will not impact your credit.
PantherX
Site Moderator
Posts: 7020
Joined: Wed Dec 23, 2009 9:33 am
Hardware configuration: V7.6.21 -> Multi-purpose 24/7
Windows 10 64-bit
CPU:2/3/4/6 -> Intel i7-6700K
GPU:1 -> Nvidia GTX 1080 Ti
§
Retired:
2x Nvidia GTX 1070
Nvidia GTX 675M
Nvidia GTX 660 Ti
Nvidia GTX 650 SC
Nvidia GTX 260 896 MB SOC
Nvidia 9600GT 1 GB OC
Nvidia 9500M GS
Nvidia 8800GTS 320 MB

Intel Core i7-860
Intel Core i7-3840QM
Intel i3-3240
Intel Core 2 Duo E8200
Intel Core 2 Duo E6550
Intel Core 2 Duo T8300
Intel Pentium E5500
Intel Pentium E5400
Location: Land Of The Long White Cloud
Contact:

Folding@Home Server Stats Page

Post by PantherX »

  • Folding@Home Server Stats
The Server Stats can be found here: https://apps.foldingathome.org/serverstats
The page is divided up into two sections and all times are shown in UTC:
  1. Project Type Stats: Information about currently available F@H Projects.
  2. Server Stats: Information about current F@H Servers.
Project Type Stats
Below is a brief definition of what the column means:
Project Type: What Project is using what FahCore.
Public Jobs: The number of WUs that can be assigned to compatible F@H clients without any flags.
Beta Jobs: The number of WUs that can be assigned to compatible F@H clients with <client-type v='beta'/>
Total Jobs: The sum of Public and Beta WUs.
Assign Rate: The number of WUs being assigned to F@H Clients per hour.


Server Stats
Below is a brief definition of what the column means:
IP: The public IP Address of the Server.
Host: The host name of the Server.
Type: The type of Server it is.
Version: The version of F@H Server software it is running.
Contact: The F@H Server owner who is the primary contact.
Assign Rate: The number of WUs being assigned to F@H Clients per hour.
Errors: Number of errors on the F@H Server and is meant for the Server maintainers.
Warnings: Number of warnings on the F@H Server and is meant for the Server maintainers.
Has CS: Does the F@H Server have a Collection Server(s) assigned to it. It is not compulsory but optional.
Status: The Status of the F@H Server.
Public Jobs: The number of WUs that can be assigned to compatible F@H clients without any flags.
Beta Jobs: The number of WUs that can be assigned to compatible F@H clients with <client-type v='beta'/>
Project Types: The types of Project the F@H Server is hosting.
Space: The amount of free space on the Server which is used to store new WUs to distribute and collect successfully completed WUs.
Uptime: How long as the Server been online for.
Last Contact: When was the last time the Server made contact with the Server Status page.


Row Colours
Light Grey: Work Servers that are actively assigning work as well as accepting returned results. The status column will show "Assign" for these.
Dark Grey: Work Servers that are only accepting returned results and do not have new work to be assigned. The status column will show "Accept" for these.
Super Dark Grey: Work Servers that are offline. The status column will show "Down" for these when the server is undergoing maintenance or is unreachable for whatever reason.

Note: If you're using <client-type v='beta'/> and not part of the F@H Beta team, support will not be provided in the forum as per the rules: viewtopic.php?f=16&t=8



Additional comments are welcomed via Private Message (PM) to me. I wish to thanks the following users who have contributed in this topic (alphabetically):
7im, anandhanju, bruce, folding_hoomer, Joe_H, mazzucha, toTOW, uncle fuzzy

Last Updated: 14 April 2020
Locked