No credit from 155.247.166.220 and 155.247.166.219

Moderators: Site Moderators, FAHC Science Team

DrBB1
Posts: 136
Joined: Wed Mar 26, 2008 12:30 am
Location: SE PA

Re: No credit from 155.247.166.220 and 155.247.166.219

Post by DrBB1 »

Brian if the server status report at http://fah-web.stanford.edu/pybeta/serverstat.html is correct, the "Connect" status for 155.247.166.220 is currently "Reject." That seems to be a separate issue from the stats, as you surmised.
========
DrBB1
ThunderRd
Posts: 78
Joined: Sun Dec 02, 2007 5:30 am
Location: Nong Khai, Thailand

Re: No credit from 155.247.166.220 and 155.247.166.219

Post by ThunderRd »

My error looks a bit different; I have not seen anyone indicate this 'Received short response, expected 512 bytes, got 13'

Code: Select all

10:42:00:WU02:FS00:0xa7:    OS Arch: AMD64
10:42:00:WU02:FS00:0xa7:********************************************************************************
10:42:00:WU02:FS00:0xa7:Project: 13747 (Run 145, Clone 4, Gen 10)
10:42:00:WU02:FS00:0xa7:Unit: 0x0000000a0002894b59d561b8d5507714
10:42:00:WU02:FS00:0xa7:Digital signatures verified
10:42:00:WU02:FS00:0xa7:Calling: mdrun -s frame10.tpr -o frame10.trr -cpi state.cpt -cpt 15 -nt 4
10:42:00:WU02:FS00:0xa7:Steps: first=25000000 total=2500000
10:42:00:WU02:FS00:0xa7:Completed 2035942 out of 2500000 steps (81%)
10:42:31:WU01:FS00:Upload 4.44%
10:42:35:WARNING:WU01:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 13
10:42:35:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:14007 run:0 clone:92 gen:9 core:0xa4 unit:0x0000000b0002894c59e4d0b678c33534
10:42:35:WU01:FS00:Uploading 4.22MiB to 155.247.166.220
10:42:35:WU01:FS00:Connecting to 155.247.166.220:8080
10:42:35:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
10:42:35:WU01:FS00:Connecting to 155.247.166.220:80
10:42:36:WU02:FS00:0xa7:Completed 2050000 out of 2500000 steps (82%)
10:43:09:WU01:FS00:Upload 4.44%
10:43:13:WARNING:WU01:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 13
10:43:35:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:14007 run:0 clone:92 gen:9 core:0xa4 unit:0x0000000b0002894c59e4d0b678c33534
10:43:35:WU01:FS00:Uploading 4.22MiB to 155.247.166.220
10:43:35:WU01:FS00:Connecting to 155.247.166.220:8080
10:43:35:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
10:43:35:WU01:FS00:Connecting to 155.247.166.220:80
10:43:45:WU02:FS00:0xa7:Completed 2075000 out of 2500000 steps (83%)
10:44:09:WU01:FS00:Upload 4.44%
10:44:13:WARNING:WU01:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 13
10:45:03:WU02:FS00:0xa7:Completed 2100000 out of 2500000 steps (84%)
10:45:12:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:14007 run:0 clone:92 gen:9 core:0xa4 unit:0x0000000b0002894c59e4d0b678c33534
10:45:12:WU01:FS00:Uploading 4.22MiB to 155.247.166.220
10:45:12:WU01:FS00:Connecting to 155.247.166.220:8080
10:45:13:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
10:45:13:WU01:FS00:Connecting to 155.247.166.220:80
10:46:06:WU02:FS00:0xa7:Completed 2125000 out of 2500000 steps (85%)
10:47:11:WU02:FS00:0xa7:Completed 2150000 out of 2500000 steps (86%)
Joe_H
Site Admin
Posts: 7857
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: No credit from 155.247.166.220 and 155.247.166.219

Post by Joe_H »

@absolutefunk WU's always get returned to the WS they came from first, they only go to the designated CS if not able to upload to the WS. Continuation of the log should show the client sending the WU back to 155.247.166.219 after trying 155.247.166.220 for most projects running at Temple. If your client shows 0.0.0.0 for the CS in FAHControl, then the WU will have to wait for 155.247.166.220 to come back online for the WU to be accepted.

As mentioned, 155.247.166.220 is currently down and not accepting connections. This is a different issue than the stats not being collected for the database.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Joe_H
Site Admin
Posts: 7857
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: No credit from 155.247.166.220 and 155.247.166.219

Post by Joe_H »

ThunderRd wrote:My error looks a bit different; I have not seen anyone indicate this 'Received short response, expected 512 bytes, got 13'
That error message usually indicates a connection being blocked somewhere between the client on your system and the server it is connecting with. Most often it has been caused by anti-malware apps or firewall settings on the system or network the client is located on. But it could be elsewhere. If you are not getting this error message on other client connections to the servers, it might be related to whatever caused the 155.247.166.220 WS to go down overnight.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
GreyWhiskers
Posts: 660
Joined: Mon Oct 25, 2010 5:57 am
Hardware configuration: a) Main unit
Sandybridge in HAF922 w/200 mm side fan
--i7 2600K@4.2 GHz
--ASUS P8P67 DeluxeB3
--4GB ADATA 1600 RAM
--750W Corsair PS
--2Seagate Hyb 750&500 GB--WD Caviar Black 1TB
--EVGA 660GTX-Ti FTW - Signature 2 GPU@ 1241 Boost
--MSI GTX560Ti @900MHz
--Win7Home64; FAH V7.3.2; 327.23 drivers

b) 2004 HP a475c desktop, 1 core Pent 4 HT@3.2 GHz; Mem 2GB;HDD 160 GB;Zotac GT430PCI@900 MHz
WinXP SP3-32 FAH v7.3.6 301.42 drivers - GPU slot only

c) 2005 Toshiba M45-S551 laptop w/2 GB mem, 160GB HDD;Pent M 740 CPU @ 1.73 GHz
WinXP SP3-32 FAH v7.3.6 [Receiving Core A4 work units]
d) 2011 lappy-15.6"-1920x1080;i7-2860QM,2.5;IC Diamond Thermal Compound;GTX 560M 1,536MB u/c@700;16GB-1333MHz RAM;HDD:500GBHyb w/ 4GB SSD;Win7HomePrem64;320.18 drivers FAH 7.4.2ß
Location: Saratoga, California USA

Re: No credit from 155.247.166.220 and 155.247.166.219

Post by GreyWhiskers »

Just to pile on, at least since the Stanford stats server went down at Thanksgiving time, I've been missing ALL credits from my CPU slots. The culprit in my case is the server at 155.247.166.219.

As the Temple IT folks are working on the firewall issue, I presume that when that happens, the whole logjam of credits will be forwarded to Stanford's stats servers, and the third party stats servers like EOC and Kakao will be able to report them as lump sums.

I've been running CPU slots on two of my Core i7 machines, both with AVX enabled, and one GPU enabled slot. Since the Stanford stats server came back up, I've been credited (as reported in my EOC stats) ONLY with results from the GPU folding, and none of the CPU folding.

FYI, I've had no folding or error issues at all in this CPU folding. I finish and upload a WU, the log reports the credit, I get a new CPU WU, fold it, complete and upload it, with the FAH control log reporting credit, etc., etc.

Here's a sample over the last couple of days of the completions and credits from the FAH Control log for one of the two computers.

Note that I was pruning the lines from the log I wanted to display here, and didn't include the IP address of the collection server in the first two instances. For the subsequent instances, it was included. Also note the missing items are a combo of A4 and A7 core WUs.
4:27:34 :WU00 :FS00 :Sending unit results: id:00 state:SEND error:NO_ERROR project:13739 run:154 clone:4 gen:7 core:0xa7 unit:0x000000070002894b59d5d7fdd885317e
4:27:36 :WU00 :FS00 :Final credit estimate, 1622.00 points

5:53:14 :WU01 :FS00 :Sending unit results: id:01 state:SEND error:NO_ERROR project:13743 run:11 clone:5 gen:4 core:0xa7 unit:0x000000040002894b59d5a38b00fbcdad
5:53:19 :WU01 :FS00 :Final credit estimate, 1659.00 points

12:37:24 :WU00 :FS00 :Sending unit results: id:00 state:SEND error:NO_ERROR project:8633 run:2 clone:606 gen:12 core:0xa4 unit:0x0000000f0002894b57f6f46806e6fe2b
12:37:24 :WU00 :FS00 :Uploading 3.22MiB to 155.247.166.219
12:37:32 :WU00 :FS00 :Final credit estimate, 4698.00 points

14:04:02 :WU01 :FS00 :Sending unit results: id:01 state:SEND error:NO_ERROR project:13744 run:122 clone:5 gen:0 core:0xa7 unit:0x000000000002894b59d58ba4e7b76a70
14:04:02 :WU01 :FS00 :Uploading 1.66MiB to 155.247.166.219
14:04:06 :WU01 :FS00 :Final credit estimate, 1619.00 points

20:46:01 :WU00 :FS00 :Sending unit results: id:00 state:SEND error:NO_ERROR project:8632 run:5 clone:215 gen:7 core:0xa4 unit:0x000000090002894b57f6f5bdd7841ccc
20:46:01 :WU00 :FS00 :Uploading 2.45MiB to 155.247.166.219
20:46:04 :WU00 :FS00 :Final credit estimate, 4644.00 points

22:09:50 :WU01 :FS00 :Sending unit results: id:01 state:SEND error:NO_ERROR project:13740 run:59 clone:4 gen:5 core:0xa7 unit:0x000000060002894b59d631776ececd66
22:09:50 :WU01 :FS00 :Uploading 1.66MiB to 155.247.166.219
22:09:55 :WU01 :FS00 :Final credit estimate, 1645.00 points
parkut
Posts: 364
Joined: Tue Feb 12, 2008 7:33 am
Hardware configuration: Running exclusively Linux headless blades. All are dedicated crunching machines.
Location: SE Michigan, USA

Failed to send : Failed to connect to 155.247.166.220:80

Post by parkut »

I have (4) machines that cannot connect to this server, FAHControl status tab is showing collection server: 0.0.0.0

15:38:26:WARNING:WU00:FS00:WorkServer connection failed on port 8080 trying 80
15:40:34:WARNING:WU00:FS00:Exception: Failed to send results to work server: Failed to connect to 155.247.166.220:80: Connection timed out

project:8652 run:2296 clone:0 gen:11
project:8642 run:9696 clone:1 gen:21
project:14012 run:0 clone:311 gen:3
project:13786 run:4 clone:48 gen:8
montemac
Posts: 41
Joined: Wed Oct 10, 2012 11:49 am
Hardware configuration: Dell XPS 13 9350, Signature Ultrabook, I7-6500U, 2.5 Ghz, Win 10 Home
Toshiba PORTEGE Z935-ST4N03 Ultrabook, I5-3370U, 1.8 Ghz, Win 8.1
EMachine ET1331 desktop, Ath. II X2 215, 2.7 Ghz, Win 7 Home Prem.
Lenovo Thinkpad, Int. 2Duo CPU, P8600, 2.4 Ghz, Vista Home Prem.
Location: Richmond, VA

Re: No credit from 155.247.166.220 and 155.247.166.219

Post by montemac »

Does this mean anything good? Just copied from the Server Status database:
14 155.247.166.219 vav3 vvoelz SMP full Accepting
15 155.247.166.220 vav4 vvoelz SMP full Accepting

Edit: Just noticed, nothing between the words "Accepting" and the last column, "OS_Weight_Program_Port" whatever that means.
Folding on 4 pc's
Image
Joe_H
Site Admin
Posts: 7857
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: No credit from 155.247.166.220 and 155.247.166.219

Post by Joe_H »

montemac wrote:Does this mean anything good? Just copied from the Server Status database:
14 155.247.166.219 vav3 vvoelz SMP full Accepting
15 155.247.166.220 vav4 vvoelz SMP full Accepting

Edit: Just noticed, nothing between the words "Accepting" and the last column, "OS_Weight_Program_Port" whatever that means.
Yes, that is good. The only column that we are looking to see have data in again is the one headed as WUs Rcv. Many of the rest no longer apply, they date back to older versions of the Work Server software and the information supplied to the Server Status page by those versions.

The number in Wus Rcv column tracks how many WU's that the server has credit information ready to be uploaded to the stats database. The collection script runs once an hour to pick up the log with the credits. Once the problem with that connection is resolved, that column should start showing information again and WU credits show show up from these two servers.

Off topic a bit, but that last column lists which OS's a project will be assigned to, the priority, and the ports over which they will be assigned. Hover over the blue "i" at the top of the column and it will give you information about the entries there.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Joe_H
Site Admin
Posts: 7857
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: Failed to send : Failed to connect to 155.247.166.220:80

Post by Joe_H »

parkut wrote:I have (4) machines that cannot connect to this server, FAHControl status tab is showing collection server: 0.0.0.0

15:38:26:WARNING:WU00:FS00:WorkServer connection failed on port 8080 trying 80
15:40:34:WARNING:WU00:FS00:Exception: Failed to send results to work server: Failed to connect to 155.247.166.220:80: Connection timed out

project:8652 run:2296 clone:0 gen:11
project:8642 run:9696 clone:1 gen:21
project:14012 run:0 clone:311 gen:3
project:13786 run:4 clone:48 gen:8
This WS was restarted earlier today. Your WU's should upload now if they have not already done so. I had some waiting to upload, they had already done so when I checked around 1 PM EST.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
DrBB1
Posts: 136
Joined: Wed Mar 26, 2008 12:30 am
Location: SE PA

Re: No credit from 155.247.166.220 and 155.247.166.219

Post by DrBB1 »

FWIW, I don't believe any of the WUs I completed that were sent to these servers have (yet) been credited, although the issue was apparently fixed over 24 hours ago. Is this a problem to investigate, or is this simply an expected lag in receiving credit? If the latter, about how long should it take before everything is all caught up?
========
DrBB1
kofther
Posts: 9
Joined: Thu Nov 30, 2017 1:00 pm

Re: No credit from 155.247.166.220 and 155.247.166.219

Post by kofther »

Are there 2 issues being referenced in this thread? Is the Temple firewall issue been resolved?
Joe_H
Site Admin
Posts: 7857
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: No credit from 155.247.166.220 and 155.247.166.219

Post by Joe_H »

kofther wrote:Are there 2 issues being referenced in this thread? Is the Temple firewall issue been resolved?
Yes. This topic started out about no credits from the Temple servers, which is possibly due to firewall issues. That has not been resolved yet.

A few posts back a problem uploading to one of the two servers there was brought up, and it was asked if that was related to the first problem. It might have been, all that is certain is that the server stopped accepting connections until it was restarted.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
kofther
Posts: 9
Joined: Thu Nov 30, 2017 1:00 pm

Re: No credit from 155.247.166.220 and 155.247.166.219

Post by kofther »

Thanks Joe. Is there an estimate fix date or update for the (what I'll call) the firewall issue?
JimboPalmer
Posts: 2573
Joined: Mon Feb 16, 2009 4:12 am
Location: Greenwood MS USA

Re: No credit from 155.247.166.220 and 155.247.166.219

Post by JimboPalmer »

I do not even think they have determined if Temple is blocking sending, or Stanford is blocking receiving. Just a lot of IT departments potentially pointing fingers until they resolve that.
Tsar of all the Rushers
I tried to remain childlike, all I achieved was childish.
A friend to those who want no friends
montemac
Posts: 41
Joined: Wed Oct 10, 2012 11:49 am
Hardware configuration: Dell XPS 13 9350, Signature Ultrabook, I7-6500U, 2.5 Ghz, Win 10 Home
Toshiba PORTEGE Z935-ST4N03 Ultrabook, I5-3370U, 1.8 Ghz, Win 8.1
EMachine ET1331 desktop, Ath. II X2 215, 2.7 Ghz, Win 7 Home Prem.
Lenovo Thinkpad, Int. 2Duo CPU, P8600, 2.4 Ghz, Vista Home Prem.
Location: Richmond, VA

Re: No credit from 155.247.166.220 and 155.247.166.219

Post by montemac »

It would be nice if the Temple and Stanford people were in on this conversation.
Folding on 4 pc's
Image
Post Reply