Folding behind a Proxy Annoying Issue

Moderators: Site Moderators, PandeGroup

Folding behind a Proxy Annoying Issue

Postby hhherby » Wed Jan 11, 2017 2:49 pm

So the issue that I'm running into when getting Work Units from the WS is that the WS that I get assigned to tends to return HTTP_BAD_GATEWAY. Here is a list of WS that I have issues with:
171.67.108.105
171.67.108.159
140.163.4.231
140.163.4.242
140.163.4.243
140.163.4.244
140.163.4.245

Work Server that fails to assign a work unit
171.67.108.159

Work Servers that do work:
171.64.65.84
171.64.65.92
171.67.108.101
171.67.108.155

The coloration that I have realized here is that the WSs that are returning HTTP_BAD_GATEWAY do not serve on port 80, only port 8080. The WSs that I do get work from serve both port 80 and 8080. The server that doesn't assign a work unit really only serves on port 8080, but when you punch in the IP address for that WS, you get the Apache2 Debian Default Page.

For the WSs that I get a HTTP_BAD_GATEWAY, I can punch in their IP:8080 in the web browser and I get the folding logo. So I'm assuming the proxy is not blocking port 8080.

The only reason I say something is because of the following news article that was posted.
Code: Select all
https://folding.stanford.edu/home/fixes-for-recent-fah-server-outage/

Where you state your failover. This doesn't seem to be followed if I'm getting assignments from port 8080. Meaning that the client is asking for the assignments on port 8080 but asking for the work on port 80.

Also another issue that might be of clue is when I setup the Proxy tab, I have to specify port 9090. Not sure if this plays a role on what port is used or not. Here is an example of the log file:

Code: Select all
14:12:11:WU02:FS01:Connecting to proxy.something.com:9090
14:12:11:WU02:FS01:Assigned to work server 171.67.108.159
14:12:11:WU02:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GM204 [GeForce GTX 980] from 171.67.108.159
14:12:11:WU02:FS01:Connecting to proxy.something.com:9090
14:12:11:ERROR:WU02:FS01:Exception: Server did not assign work unit
14:12:12:WU02:FS01:Connecting to proxy.something.com:9090
14:12:12:WU02:FS01:Assigned to work server 140.163.4.244
14:12:12:WU02:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GM204 [GeForce GTX 980] from 140.163.4.244
14:12:12:WU02:FS01:Connecting to proxy.something.com:9090
14:12:13:ERROR:WU02:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
14:12:17:WU01:FS01:0x21:Saving result file logfile_01.txt
14:12:17:WU01:FS01:0x21:Saving result file checkpointState.xml
14:12:21:WU01:FS01:0x21:Saving result file checkpt.crc
14:12:21:WU01:FS01:0x21:Saving result file log.txt
14:12:21:WU01:FS01:0x21:Saving result file positions.xtc
14:12:23:WU01:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
14:12:23:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
14:12:23:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13110 run:89 clone:0 gen:514 core:0x21 unit:0x00000172ab436c65577170b765b74928
14:12:23:WU01:FS01:Uploading 13.94MiB to 171.67.108.101
14:12:23:WU01:FS01:Connecting to proxy.something.com:9090
14:12:29:WU01:FS01:Upload complete
14:12:29:WU01:FS01:Server responded WORK_ACK (400)
14:12:29:WU01:FS01:Final credit estimate, 21396.00 points
14:12:29:WU01:FS01:Cleaning up
14:13:12:WU02:FS01:Connecting to proxy.something.com:9090
14:13:12:WU02:FS01:Assigned to work server 171.64.65.92
14:13:12:WU02:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 171.64.65.92
14:13:12:WU02:FS01:Connecting to proxy.something.com:9090
14:13:15:WU02:FS01:Downloading 2.52MiB
14:13:16:WU02:FS01:Download complete
hhherby
 
Posts: 12
Joined: Thu Jan 05, 2017 9:30 pm

Re: Folding behind a Proxy Annoying Issue

Postby bruce » Thu Jan 12, 2017 12:47 am

How have you set up the proxy in FAH's configuration? Please post the first few pages of your log (per the instructions in my Sig.)
bruce
Site Admin
 
Posts: 20857
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Folding behind a Proxy Annoying Issue

Postby hhherby » Thu Jan 12, 2017 3:42 am

Code: Select all
*********************** Log Started 2017-01-03T18:03:31Z ***********************
18:03:31:************************* Folding@home Client *************************
18:03:31:      Website: http://folding.stanford.edu/
18:03:31:    Copyright: (c) 2009-2014 Stanford University
18:03:31:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
18:03:31:         Args:
18:03:31:       Config: C:/Users/jjherbra/AppData/Roaming/FAHClient/config.xml
18:03:31:******************************** Build ********************************
18:03:31:      Version: 7.4.4
18:03:31:         Date: Mar 4 2014
18:03:31:         Time: 20:26:54
18:03:31:      SVN Rev: 4130
18:03:31:       Branch: fah/trunk/client
18:03:31:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
18:03:31:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
18:03:31:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
18:03:31:     Platform: win32 XP
18:03:31:         Bits: 32
18:03:31:         Mode: Release
18:03:31:******************************* System ********************************
18:03:31:          CPU: Intel(R) Xeon(R) CPU W3520 @ 2.67GHz
18:03:31:       CPU ID: GenuineIntel Family 6 Model 26 Stepping 5
18:03:31:         CPUs: 4
18:03:31:       Memory: 4.00GiB
18:03:31:  Free Memory: 1.90GiB
18:03:31:      Threads: WINDOWS_THREADS
18:03:31:   OS Version: 6.1
18:03:31:  Has Battery: false
18:03:31:   On Battery: false
18:03:31:   UTC Offset: -6
18:03:31:          PID: 1948
18:03:31:          CWD: C:/Users/jjherbra/AppData/Roaming/FAHClient
18:03:31:           OS: Windows 7 Enterprise
18:03:31:      OS Arch: AMD64
18:03:31:         GPUs: 1
18:03:31:        GPU 0: NVIDIA:5 GM204 [GeForce GTX 980]
18:03:31:         CUDA: 5.2
18:03:31:  CUDA Driver: 8000
18:03:31:Win32 Service: false
18:03:31:***********************************************************************
18:03:31:<config>
18:03:31:  <!-- HTTP Server -->
18:03:31:  <allow v='131.198.0.0/16'/>
18:03:31:
18:03:31:  <!-- Network -->
18:03:31:  <proxy v='proxy.something.com:9090'/>
18:03:31:  <proxy-enable v='true'/>
18:03:31:
18:03:31:  <!-- Remote Command Server -->
18:03:31:  <command-allow-no-pass v='131.198.0.0/16'/>
18:03:31:
18:03:31:  <!-- Slot Control -->
18:03:31:  <power v='full'/>
18:03:31:
18:03:31:  <!-- User Information -->
18:03:31:  <passkey v='********************************'/>
18:03:31:  <team v='111065'/>
18:03:31:  <user v='hhherby'/>
18:03:31:
18:03:31:  <!-- Folding Slots -->
18:03:31:  <slot id='1' type='GPU'>
18:03:31:    <next-unit-percentage v='100'/>
18:03:31:  </slot>
18:03:31:  <slot id='0' type='CPU'>
18:03:31:    <cpus v='3'/>
18:03:31:    <next-unit-percentage v='100'/>
18:03:31:  </slot>
18:03:31:</config>
18:03:31:Trying to access database...
18:03:32:Successfully acquired database lock
18:03:32:Enabled folding slot 01: READY gpu:0:GM204 [GeForce GTX 980]
18:03:32:Enabled folding slot 00: READY cpu:3
18:03:32:WU02:FS00:Starting
18:03:32:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/jjherbra/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 02 -suffix 01 -version 704 -lifeline 1948 -checkpoint 15 -np 3
18:03:32:WU02:FS00:Started FahCore on PID 1324
18:03:32:WU02:FS00:Core PID:7316
18:03:32:WU02:FS00:FahCore 0xa4 started
18:03:32:WU00:FS01:Connecting to proxy.something.com:9090
18:03:32:WU02:FS00:0xa4:
18:03:32:WU02:FS00:0xa4:*------------------------------*
18:03:32:WU02:FS00:0xa4:Folding@Home Gromacs GB Core
18:03:32:WU02:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
18:03:32:WU02:FS00:0xa4:
18:03:32:WU02:FS00:0xa4:Preparing to commence simulation
18:03:32:WU02:FS00:0xa4:- Ensuring status. Please wait.
18:03:33:WU00:FS01:Assigned to work server 140.163.4.231
18:03:33:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 140.163.4.231
18:03:33:WU00:FS01:Connecting to proxy.something.com:9090
18:03:33:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
18:03:33:WU00:FS01:Connecting to proxy.something.com:9090
18:03:34:WU00:FS01:Assigned to work server 140.163.4.231
18:03:34:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 140.163.4.231
18:03:34:WU00:FS01:Connecting to proxy.something.com:9090
18:03:34:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
18:03:41:WU02:FS00:0xa4:- Looking at optimizations...
18:03:41:WU02:FS00:0xa4:- Working with standard loops on this execution.
18:03:41:WU02:FS00:0xa4:- Previous termination of core was improper.
18:03:41:WU02:FS00:0xa4:- Files status OK
18:03:41:WU02:FS00:0xa4:- Expanded 825931 -> 1402860 (decompressed 169.8 percent)
18:03:41:WU02:FS00:0xa4:Called DecompressByteArray: compressed_data_size=825931 data_size=1402860, decompressed_data_size=1402860 diff=0
18:03:41:WU02:FS00:0xa4:- Digital signature verified
18:03:41:WU02:FS00:0xa4:
18:03:41:WU02:FS00:0xa4:Project: 9038 (Run 351, Clone 2, Gen 535)
18:03:41:WU02:FS00:0xa4:
18:03:41:WU02:FS00:0xa4:Entering M.D.
18:03:47:WU02:FS00:0xa4:Using Gromacs checkpoints
18:03:47:WU02:FS00:0xa4:Mapping NT from 3 to 3
18:03:48:WU02:FS00:0xa4:Resuming from checkpoint
18:03:48:WU02:FS00:0xa4:Verified 02/wudata_01.log
18:03:48:WU02:FS00:0xa4:Verified 02/wudata_01.trr
18:03:48:WU02:FS00:0xa4:Verified 02/wudata_01.xtc
18:03:48:WU02:FS00:0xa4:Verified 02/wudata_01.edr
18:03:48:WU02:FS00:0xa4:Completed 154270 out of 250000 steps  (61%)
18:04:33:WU00:FS01:Connecting to proxy.something.com:9090
18:04:34:WU00:FS01:Assigned to work server 140.163.4.244
18:04:34:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 140.163.4.244
18:04:34:WU00:FS01:Connecting to proxy.something.com:9090
18:04:34:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
18:04:38:WU02:FS00:0xa4:Completed 155000 out of 250000 steps  (62%)
18:05:28:FS01:Paused
18:05:31:FS01:Unpaused
18:05:33:WU00:FS01:Connecting to proxy.something.com:9090
18:05:34:WU00:FS01:Assigned to work server 171.67.108.102
18:05:34:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 171.67.108.102
18:05:34:WU00:FS01:Connecting to proxy.something.com:9090
18:05:34:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
18:07:11:WU00:FS01:Connecting to proxy.something.com:9090
18:07:11:WU00:FS01:Assigned to work server 171.67.108.105
18:07:11:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 171.67.108.105
18:07:11:WU00:FS01:Connecting to proxy.something.com:9090
18:07:12:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
18:07:27:WU02:FS00:0xa4:Completed 157500 out of 250000 steps  (63%)
18:09:48:WU00:FS01:Connecting to proxy.something.com:9090
18:09:48:WU00:FS01:Assigned to work server 171.64.65.84
18:09:48:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 171.64.65.84
18:09:48:WU00:FS01:Connecting to proxy.something.com:9090
18:09:51:WU00:FS01:Downloading 2.52MiB
18:09:52:WU00:FS01:Download complete
18:09:52:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:9188 run:1 clone:27 gen:240 core:0x21 unit:0x0000018bab40415457cb2b4f25dbe3c2
18:09:52:WU00:FS01:Starting
18:09:52:WU00:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/jjherbra/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 00 -suffix 01 -version 704 -lifeline 1948 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
18:09:52:WU00:FS01:Started FahCore on PID 6296
18:09:52:WU00:FS01:Core PID:7048
18:09:52:WU00:FS01:FahCore 0x21 started
18:09:53:WU00:FS01:0x21:*********************** Log Started 2017-01-03T18:09:52Z ***********************
18:09:53:WU00:FS01:0x21:Project: 9188 (Run 1, Clone 27, Gen 240)
18:09:53:WU00:FS01:0x21:Unit: 0x0000018bab40415457cb2b4f25dbe3c2
18:09:53:WU00:FS01:0x21:CPU: 0x00000000000000000000000000000000
18:09:53:WU00:FS01:0x21:Machine: 1
18:09:53:WU00:FS01:0x21:Reading tar file core.xml
18:09:53:WU00:FS01:0x21:Reading tar file system.xml
18:09:53:WU00:FS01:0x21:Reading tar file integrator.xml
18:09:53:WU00:FS01:0x21:Reading tar file state.xml
18:09:53:WU00:FS01:0x21:Digital signatures verified
18:09:53:WU00:FS01:0x21:Folding@home GPU Core21 Folding@home Core
18:09:53:WU00:FS01:0x21:Version 0.0.17
18:09:59:WU00:FS01:0x21:Completed 0 out of 2500000 steps (0%)
18:09:59:WU00:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
18:10:18:WU02:FS00:0xa4:Completed 160000 out of 250000 steps  (64%)
18:11:04:WU00:FS01:0x21:Completed 25000 out of 2500000 steps (1%)
hhherby
 
Posts: 12
Joined: Thu Jan 05, 2017 9:30 pm

Re: Folding behind a Proxy Annoying Issue

Postby bruce » Fri Jan 13, 2017 3:39 am

Does this represent the actual information associated with your real proxy?
Code: Select all
18:03:31:  <proxy v='proxy.something.com:9090'/>
18:03:31:  <proxy-enable v='true'/>
bruce
Site Admin
 
Posts: 20857
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Folding behind a Proxy Annoying Issue

Postby hhherby » Fri Jan 13, 2017 5:03 am

Do to social media rules set by my company, "something" represents my companies name. Other then that, it is correct. Do you need the correct name of the proxy because it can't be pinged outside the network.
hhherby
 
Posts: 12
Joined: Thu Jan 05, 2017 9:30 pm

Re: Folding behind a Proxy Annoying Issue

Postby bruce » Fri Jan 13, 2017 6:33 am

No, I don't need it. It just didn't look right. Presumably port 9090 is the right port on the real proxy name.

Yes, this is an annoying issue. The code for a proxy used to work but so few people use it that I don't know if the lack of truble reports are because nobody uses it or because your system needs to be configured differently.

If you ping proxy.something.com:9090, do you get a reply? That should also tell you the IP address associated with the proxy server. Is that address within the range of 131.198.0.0/16?

- - -

What happens if you try to run FAH without the proxy settings?
bruce
Site Admin
 
Posts: 20857
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Folding behind a Proxy Annoying Issue

Postby hhherby » Fri Jan 13, 2017 6:26 pm

Yes, I can ping proxy.something.com. I get address 10.16.4.25 for the proxy. As you can see, it is not in the range of 131.198.0.0/16 where the machine running the client is on IP 131.198.187.21. The reason for the range for "HTTP Server" and "Remote Command Server" section is so I can run the FAHControl program from computer 131.198.187.200 as a remote client. This way I can manage multiple computers from one spot under your "Remote access" tab in the FAHControl program.

If I run the folding client with the Proxy Configuration disabled, I get the following:

Code: Select all
17:53:05:WU00:FS00:Connecting to 171.67.108.45:8080
17:53:26:WARNING:WU00:FS00:Failed to get assignment from '171.67.108.45:8080': Failed to connect to 171.67.108.45:8080: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
17:53:26:WU00:FS00:Connecting to 171.64.65.35:80
17:53:48:WARNING:WU00:FS00:Failed to get assignment from '171.64.65.35:80': Failed to connect to 171.64.65.35:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
17:53:48:ERROR:WU00:FS00:Exception: Could not get an assignment
17:53:48:WU00:FS00:Connecting to 171.67.108.45:8080
17:54:09:WARNING:WU00:FS00:Failed to get assignment from '171.67.108.45:8080': Failed to connect to 171.67.108.45:8080: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
17:54:09:WU00:FS00:Connecting to 171.64.65.35:80
17:54:16:FS00:Paused
17:54:30:WARNING:WU00:FS00:Failed to get assignment from '171.64.65.35:80': Failed to connect to 171.64.65.35:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
17:54:30:ERROR:WU00:FS00:Exception: Could not get an assignment


So if I want to get out to the internet, I need to have the proxy section enabled.
hhherby
 
Posts: 12
Joined: Thu Jan 05, 2017 9:30 pm

Re: Folding behind a Proxy Annoying Issue

Postby bruce » Fri Jan 13, 2017 7:06 pm

"127.0.0.1,131.198.0.0/16,10.16.4.25" should work, (or you can use a more general range for the 10.x.x.x network if that's needed.)
bruce
Site Admin
 
Posts: 20857
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Folding behind a Proxy Annoying Issue

Postby hhherby » Fri Jan 13, 2017 9:59 pm

So I have updated one of my clients to the following as suggested I believe:

Code: Select all
*********************** Log Started 2017-01-13T20:20:28Z ***********************
20:20:28:************************* Folding@home Client *************************
20:20:28:      Website: http://folding.stanford.edu/
20:20:28:    Copyright: (c) 2009-2014 Stanford University
20:20:28:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
20:20:28:         Args:
20:20:28:       Config: C:/Users/jjherbra/AppData/Roaming/FAHClient/config.xml
20:20:28:******************************** Build ********************************
20:20:28:      Version: 7.4.4
20:20:28:         Date: Mar 4 2014
20:20:28:         Time: 20:26:54
20:20:28:      SVN Rev: 4130
20:20:28:       Branch: fah/trunk/client
20:20:28:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
20:20:28:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
20:20:28:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
20:20:28:     Platform: win32 XP
20:20:28:         Bits: 32
20:20:28:         Mode: Release
20:20:28:******************************* System ********************************
20:20:28:          CPU: Intel(R) Xeon(R) CPU W3520 @ 2.67GHz
20:20:28:       CPU ID: GenuineIntel Family 6 Model 26 Stepping 5
20:20:28:         CPUs: 4
20:20:28:       Memory: 4.00GiB
20:20:28:  Free Memory: 2.19GiB
20:20:28:      Threads: WINDOWS_THREADS
20:20:28:   OS Version: 6.1
20:20:28:  Has Battery: false
20:20:28:   On Battery: false
20:20:28:   UTC Offset: -6
20:20:28:          PID: 10780
20:20:28:          CWD: C:/Users/jjherbra/AppData/Roaming/FAHClient
20:20:28:           OS: Windows 7 Enterprise
20:20:28:      OS Arch: AMD64
20:20:28:         GPUs: 1
20:20:28:        GPU 0: NVIDIA:5 GM204 [GeForce GTX 980]
20:20:28:         CUDA: 5.2
20:20:28:  CUDA Driver: 8000
20:20:28:Win32 Service: false
20:20:28:***********************************************************************
20:20:28:<config>
20:20:28:  <!-- HTTP Server -->
20:20:28:  <allow v='127.0.0.1,131.198.0.0/16,10.16.4.25'/>
20:20:28:
20:20:28:  <!-- Network -->
20:20:28:  <proxy v='proxy.something.com:9090'/>
20:20:28:  <proxy-enable v='true'/>
20:20:28:
20:20:28:  <!-- Remote Command Server -->
20:20:28:  <command-allow-no-pass v='127.0.0.1,131.198.0.0/16,10.16.4.25'/>
20:20:28:
20:20:28:  <!-- Slot Control -->
20:20:28:  <power v='full'/>
20:20:28:
20:20:28:  <!-- User Information -->
20:20:28:  <passkey v='********************************'/>
20:20:28:  <team v='111065'/>
20:20:28:  <user v='hhherby'/>
20:20:28:
20:20:28:  <!-- Folding Slots -->
20:20:28:  <slot id='1' type='GPU'/>
20:20:28:  <slot id='0' type='CPU'>
20:20:28:    <cpus v='3'/>
20:20:28:    <next-unit-percentage v='100'/>
20:20:28:  </slot>
20:20:28:</config>
20:20:28:Trying to access database...
20:20:28:Successfully acquired database lock
20:20:28:Enabled folding slot 01: READY gpu:0:GM204 [GeForce GTX 980]
20:20:28:Enabled folding slot 00: READY cpu:3
20:20:28:WU01:FS00:Starting
20:20:28:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/jjherbra/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 704 -lifeline 10780 -checkpoint 15 -np 3
20:20:28:WU01:FS00:Started FahCore on PID 2652
20:20:28:WU01:FS00:Core PID:9204
20:20:28:WU01:FS00:FahCore 0xa4 started
20:20:28:WU02:FS01:Starting
20:20:28:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/jjherbra/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 02 -suffix 01 -version 704 -lifeline 10780 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
20:20:28:WU02:FS01:Started FahCore on PID 2496
20:20:28:WU02:FS01:Core PID:6232
20:20:28:WU02:FS01:FahCore 0x21 started


But it still has issues with Work Servers that don't serve on port 80

Code: Select all
21:43:44:WU02:FS01:0x21:Completed 2450000 out of 2500000 steps (98%)
21:44:51:WU02:FS01:0x21:Completed 2475000 out of 2500000 steps (99%)
21:44:52:WU01:FS01:Connecting to proxy.something.com:9090
21:44:53:WU01:FS01:Assigned to work server 171.67.108.105
21:44:53:WU01:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GM204 [GeForce GTX 980] from 171.67.108.105
21:44:53:WU01:FS01:Connecting to proxy.something.com:9090
21:44:53:ERROR:WU01:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
21:44:53:WU01:FS01:Connecting to proxy.something.com:9090
21:44:54:WU01:FS01:Assigned to work server 171.64.65.84
21:44:54:WU01:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GM204 [GeForce GTX 980] from 171.64.65.84
21:44:54:WU01:FS01:Connecting to proxy.something.com:9090
21:44:57:WU01:FS01:Downloading 3.18MiB
21:44:58:WU01:FS01:Download complete
21:44:58:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:9192 run:1 clone:10 gen:151 core:0x21 unit:0x000000e9ab40415457cb2d38cd1e70c0
hhherby
 
Posts: 12
Joined: Thu Jan 05, 2017 9:30 pm

Re: Folding behind a Proxy Annoying Issue

Postby bruce » Fri Jan 13, 2017 10:44 pm

FAH uses either port 8080 (first choice) or port 80. Proxy servers may pass requests to port 8080 or they may only work with port 80. (That's a configuration setting made by your proxy administrator.)

When there is no proxy and a connection is attempted to port 8080 on a FAH server, that connection may fail if port 8080 is not supported. FAH is designed to retry that connection attempt to port 80. In the no-proxy case, I've never seen the error HTTP_BAD_GATEWAY so that must be coming from your proxy server. Apparently FAHClient doesn't have code to recover from that type of error.

As I said, the FAHClient and the FAH servers will always try port 8080 first and revert to port 80 if it recognizes it needs to do that. In most cases, port 80 isn't essential so if that port fails, it might not be noticed. Nevertheless, if you report that port 80 is not available on, say, 171.67.108.105, we can get that port re-enabled. From your log, it seems that that port needs to be reset but 171.64.65.84:80 is working.
bruce
Site Admin
 
Posts: 20857
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Folding behind a Proxy Annoying Issue

Postby hhherby » Fri Jan 13, 2017 11:17 pm

Will, the only reason I started this post was as I stated at the beginning, the WSs I receive the HTTP_BAD_GATEWAY response are just WSs that don't have port 80 enabled. But if I put for example "171.67.108.105:8080" in my web browser here at work, I get the folding logo in my web browser. So I'm assuming port 8080 request was forwarded. If I don't use port 8080 in the web browser, then I get a Cannot Connect error in my web browser. So I was on the subsumption that the proxy server was passing the request on port 8080 for my web browser and the Assignment Server.

Then for an oddity, as I stated at the beginning of the post, WS 171.67.108.159 only has port 8080 enabled, but when you don't use port 8080 in the web browser, you get the Apache2 Debian Default Page. For when the client asks for an assignment from this WS, I don't get the HTTP_BAD_GATEWAY response, but instead, I get "Work Server fails to assign a work unit". So I was assuming that some kind of response made it to the folding client, but wasn't correct.

Nevertheless, if I need to make request to get port 80 re-enabled, do I do that here? From the beginning of the post, I'm having issues with seven WSs.

Thanks for all you help!
hhherby
 
Posts: 12
Joined: Thu Jan 05, 2017 9:30 pm

Re: Folding behind a Proxy Annoying Issue

Postby bruce » Sat Jan 14, 2017 6:27 am

In addition to your list, servers 155.247.166.219 and 155.247.166.220 need to have port 80 redirected, too. I've asked that the owners of those servers correct those problems.
bruce
Site Admin
 
Posts: 20857
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Folding behind a Proxy Annoying Issue

Postby bruce » Sat Jan 14, 2017 6:55 am

The only servers that matter are the ones that stanford's assignment servers recognize as having work to be assigned to your system. The assignment servers will not directy you to Work Servers which have run out of work so it's not important if they happen to remain on-line. Similarly, if a problem is identified, a project may be suspended until it can be corrected and the assignment server takes that into account. If a project is being taken off-line, the server needs to stop assigning new work but it needs to remain on-line accepting completed WUss until all the WUs in flight are either returned or time-out.

You probably call them Work Servers that do work.

Note that the status changes from "full" (assigning and accepting results) to "accepting" (not assigning new work) or to " standby"

An ideal "full" server has just enough WUs to keep them all in flight without actually running out. If it's a borderline case, it will sometime have work to assign and sometimes not. There's a time delay from when the server is polled and the assignment server recognizes that change so there will be cases when you might be assigned to a work server that can't assign work but it should correct itself fairly quickly. Having a large excess number of WUs waiting to be assigned slows the project down because they're not being worked on. Nevertheless, there should be enough WUs from at least one of the work servers to satisfy the immediate demands. That can be a delicate balance because clients request new assignments at random times.

This has nothing to do with the functionality of your proxy so technically this is off-topic information.

Once we get port 80 working on the servers, I expect that you'll never have to wait very long to get an assignment, even if it doesn't come from the first server you're assigned to.
bruce
Site Admin
 
Posts: 20857
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Folding behind a Proxy Annoying Issue

Postby hhherby » Sat Jan 14, 2017 2:46 pm

Yes, this is why this issue is annoying. The clients that are folding on the 0x21 project will get stuck trying to find work from WS. Here is example:

Code: Select all
06:19:24:WU00:FS01:Connecting to proxy.something.com:9090
06:19:25:WU00:FS01:News:
06:19:25:WU00:FS01:Assigned to work server 140.163.4.242
06:19:25:WU00:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GM204 [GeForce GTX 980] from 140.163.4.242
06:19:25:WU00:FS01:Connecting to proxy.something.com:9090
06:19:25:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
06:19:25:WU00:FS01:Connecting to proxy.something.com:9090
06:19:25:WU00:FS01:News:
06:19:25:WU00:FS01:Assigned to work server 140.163.4.231
06:19:25:WU00:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GM204 [GeForce GTX 980] from 140.163.4.231
06:19:25:WU00:FS01:Connecting to proxy.something.com:9090
06:19:26:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
06:20:25:WU00:FS01:Connecting to proxy.something.com:9090
06:20:25:WU00:FS01:News:
06:20:25:WU00:FS01:Assigned to work server 140.163.4.244
06:20:25:WU00:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GM204 [GeForce GTX 980] from 140.163.4.244
06:20:25:WU00:FS01:Connecting to proxy.something.com:9090
06:20:26:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
06:20:41:WU02:FS01:0x21:Completed 2475000 out of 2500000 steps (99%)
06:21:59:WU02:FS01:0x21:Completed 2500000 out of 2500000 steps (100%)
06:22:02:WU00:FS01:Connecting to proxy.something.com:9090
06:22:03:WU00:FS01:News:
06:22:03:WU00:FS01:Assigned to work server 140.163.4.244
06:22:03:WU00:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GM204 [GeForce GTX 980] from 140.163.4.244
06:22:03:WU00:FS01:Connecting to proxy.something.com:9090
06:22:03:WU02:FS01:0x21:Saving result file logfile_01.txt
06:22:03:WU02:FS01:0x21:Saving result file checkpointState.xml
06:22:03:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
06:22:05:WU02:FS01:0x21:Saving result file checkpt.crc
06:22:05:WU02:FS01:0x21:Saving result file log.txt
06:22:05:WU02:FS01:0x21:Saving result file positions.xtc
06:22:06:WU02:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
06:22:07:WU02:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
06:22:07:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:9192 run:1 clone:85 gen:163 core:0x21 unit:0x00000107ab40415457cb2d5dd6a76012
06:22:07:WU02:FS01:Uploading 5.92MiB to 171.64.65.84
06:22:07:WU02:FS01:Connecting to proxy.something.com:9090
06:22:14:WU02:FS01:Upload complete
06:22:14:WU02:FS01:Server responded WORK_ACK (400)
06:22:14:WU02:FS01:Final credit estimate, 34210.00 points
06:22:15:WU02:FS01:Cleaning up
06:24:39:WU00:FS01:Connecting to proxy.something.com:9090
06:24:40:WU00:FS01:News:
06:24:40:WU00:FS01:Assigned to work server 140.163.4.244
06:24:40:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 140.163.4.244
06:24:40:WU00:FS01:Connecting to proxy.something.com:9090
06:24:40:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
06:28:54:WU00:FS01:Connecting to proxy.something.com:9090
06:28:54:WU00:FS01:News:
06:28:54:WU00:FS01:Assigned to work server 171.67.108.102
06:28:54:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 171.67.108.102
06:28:54:WU00:FS01:Connecting to proxy.something.com:9090
06:28:55:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
06:35:45:WU00:FS01:Connecting to proxy.something.com:9090
06:35:45:WU00:FS01:News:
06:35:45:WU00:FS01:Assigned to work server 171.67.108.159
06:35:46:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 171.67.108.159
06:35:46:WU00:FS01:Connecting to proxy.something.com:9090
06:35:46:ERROR:WU00:FS01:Exception: Server did not assign work unit
06:46:51:WU00:FS01:Connecting to proxy.something.com:9090
06:46:51:WU00:FS01:News:
06:46:51:WU00:FS01:Assigned to work server 171.67.108.159
06:46:51:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 171.67.108.159
06:46:51:WU00:FS01:Connecting to proxy.something.com:9090
06:46:51:ERROR:WU00:FS01:Exception: Server did not assign work unit
07:04:47:WU00:FS01:Connecting to proxy.something.com:9090
07:04:48:WU00:FS01:News:
07:04:48:WU00:FS01:Assigned to work server 171.67.108.105
07:04:48:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 171.67.108.105
07:04:48:WU00:FS01:Connecting to proxy.something.com:9090
07:04:48:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
07:33:50:WU00:FS01:Connecting to proxy.something.com:9090
07:33:50:WU00:FS01:News:
07:33:50:WU00:FS01:Assigned to work server 140.163.4.244
07:33:50:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 140.163.4.244
07:33:50:WU00:FS01:Connecting to proxy.something.com:9090
07:33:51:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
******************************* Date: 2017-01-14 *******************************
08:20:49:WU00:FS01:Connecting to proxy.something.com:9090
08:20:49:WU00:FS01:News:
08:20:49:WU00:FS01:Assigned to work server 171.67.108.105
08:20:49:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 171.67.108.105
08:20:49:WU00:FS01:Connecting to proxy.something.com:9090
08:20:50:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
09:36:50:WU00:FS01:Connecting to proxy.something.com:9090
09:36:50:WU00:FS01:News:
09:36:50:WU00:FS01:Assigned to work server 140.163.4.245
09:36:50:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 140.163.4.245
09:36:50:WU00:FS01:Connecting to proxy.something.com:9090
09:36:50:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
11:39:49:WU00:FS01:Connecting to proxy.something.com:9090
11:39:50:WU00:FS01:News:
11:39:50:WU00:FS01:Assigned to work server 140.163.4.231
11:39:50:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 140.163.4.231
11:39:50:WU00:FS01:Connecting to proxy.something.com:9090
11:39:50:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
******************************* Date: 2017-01-14 *******************************
14:22:26:FS01:Paused
14:22:29:FS01:Unpaused
14:22:29:WU00:FS01:Connecting to proxy.something.com:9090
14:22:30:WU00:FS01:News:
14:22:30:WU00:FS01:Assigned to work server 171.67.108.159
14:22:30:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 171.67.108.159
14:22:30:WU00:FS01:Connecting to proxy.something.com:9090
14:22:30:ERROR:WU00:FS01:Exception: Server did not assign work unit
14:24:07:WU00:FS01:Connecting to proxy.something.com:9090
14:24:07:WU00:FS01:News:
14:24:07:WU00:FS01:Assigned to work server 171.67.108.105
14:24:07:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 171.67.108.105
14:24:07:WU00:FS01:Connecting to proxy.something.com:9090
14:24:07:ERROR:WU00:FS01:Exception: 10001: Server responded: HTTP_BAD_GATEWAY
14:26:44:WU00:FS01:Connecting to proxy.something.com:9090
14:26:44:WU00:FS01:News:
14:26:44:WU00:FS01:Assigned to work server 171.64.65.84
14:26:44:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 980] from 171.64.65.84
14:26:44:WU00:FS01:Connecting to proxy.something.com:9090
14:26:48:WU00:FS01:Downloading 2.58MiB
14:26:48:WU00:FS01:Download complete


Here I lost 8 hours or so of folding time until I come in and break the time cycle of waiting.

Here are the work servers it cycled threw:
140.163.4.231
140.163.4.242
140.163.4.244
140.163.4.245
171.67.108.102
171.67.108.105
171.67.108.159
hhherby
 
Posts: 12
Joined: Thu Jan 05, 2017 9:30 pm

Re: Folding behind a Proxy Annoying Issue

Postby bruce » Sat Jan 14, 2017 6:44 pm

The first line of that log says Connecting to proxy.something.com:9090 which is not useful.
Ordinarily (without a proxy) it will say something like
Connecting to 171.67.108.45:80
Assigned to work server xxx.xxx.xxx.xxx
...where 171.67.108.45 is an assignment server and 80 is the port being used.

As I said previously, the client should always try port 8080 first and only if that connection fails will it switch to port 80. In other words there's critical information BEFORE the point where you started posting. What was the result of the PREVIOUS attempt to connect to an assignment server?

The information that you did post demonstrates that the AS that's issuing the message "News:" is assigning you to a series of port 80 connections to various WS, all of which have port 80 blocked.
Apparently the "News:" AS has been configured so that it doesn't look for connections to port 8080, only port 80, and for some reason it's send you to plfah1* servers but not to plfah2* servers.
bruce
Site Admin
 
Posts: 20857
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Next

Return to V7.4.4 Public Release Windows/Linux/MacOS X

Who is online

Users browsing this forum: No registered users and 1 guest

cron