No work for a month?

[lshurr]

Something weird's been happening for the past month. I haven't had any new work units since June 16. I didn't notice for a long time because everything's been shipshape and Bristol fashion since I installed 5.03 in 2004. I could forget about it for long periods of time because (up until now) it just runs. I only noticed when I reactivated my 2nd machine which has been down for several months. I checked folding and the status was "Attempting to get work packet." Checking my primary machine, which has been up nearly continuously for several years nothwithstanding the occasional reboot or power outage and it, too, says "Attempting to get work packet." Time to check status... Wha! Last work unit 2010-06-16 18:04:21? I think, "Maybe I should update the client," so the reactivated secondary machine gets updated to 6.23. Same result. The log file for the secondary machine is below. Have I missed something simple?

The log file on the primary machine a little different. It complains over and over that the work unit has an invalid address. If you want to see that, I'll send it in.

It's been so trouble free that I've never had to do anything until now.

Larry

--- Opening Log file [July 27 02:05:10 UTC]


# Windows CPU Systray Edition #################################################
###############################################################################

Folding@Home Client Version 6.23

http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Documents and Settings\lshurr\Application Data\Folding@home-x86


[02:07:02] - Ask before connecting: No
[02:07:02] - User name: Larry_Shurr (Team 73846)
[02:07:02] - User ID: 772E1CF44B61BE23
[02:07:02] - Machine ID: 1
[02:07:02]


--- Opening Log file [July 27 02:14:32 UTC]


# Windows CPU Systray Edition #################################################
###############################################################################

Folding@Home Client Version 6.23

http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Documents and Settings\lshurr\Application Data\Folding@home-x86


[02:14:32] - Ask before connecting: No
[02:14:32] - User name: Larry_Shurr (Team 73846)
[02:14:32] - User ID: 772E1CF44B61BE23
[02:14:32] - Machine ID: 1
[02:14:32]
[02:14:32] Work directory not found. Creating...
[02:14:32] Could not open work queue, generating new queue...
[02:14:32] Initialization complete
[02:14:32] - Preparing to get new work unit...
[02:14:32] + Attempting to get work packet
[02:14:32] - Connecting to assignment server
[02:14:33] - Successful: assigned to (171.64.65.60).
[02:14:33] + News From Folding@Home: Welcome to Folding@Home
[02:14:33] Loaded queue successfully.
[02:15:11] Opening C:\Documents and Settings\lshurr\Application Data\Folding@home-x86\MyFolding.html...
[02:18:30] + Could not connect to Work Server
[02:18:30] - Attempt #1 to get work failed, and no other work to do.
Waiting before retry.
[02:18:39] Opening http://fah-web.stanford.edu/cgi-bin/mai ... ry_Shurr...
[02:18:44] + Attempting to get work packet
[02:18:44] - Connecting to assignment server
[02:18:45] - Successful: assigned to (171.64.65.60).
[02:18:45] + News From Folding@Home: Welcome to Folding@Home
[02:18:45] Loaded queue successfully.
[02:19:20] Opening ...
[02:22:18] + Could not connect to Work Server
[02:22:18] - Attempt #2 to get work failed, and no other work to do.
Waiting before retry.
[02:22:31] + Attempting to get work packet
[02:22:31] - Connecting to assignment server
[02:22:31] - Successful: assigned to (171.64.65.60).
[02:22:31] + News From Folding@Home: Welcome to Folding@Home
[02:22:32] Loaded queue successfully.
[02:22:53] - Couldn't send HTTP request to server
[02:22:53] + Could not connect to Work Server
[02:22:53] - Attempt #3 to get work failed, and no other work to do.
Waiting before retry.
[02:23:15] + Attempting to get work packet
[02:23:15] - Connecting to assignment server
[02:23:15] - Successful: assigned to (171.64.65.60).
[02:23:15] + News From Folding@Home: Welcome to Folding@Home
[02:23:15] Loaded queue successfully.
[02:27:13] + Could not connect to Work Server
[02:27:13] - Attempt #4 to get work failed, and no other work to do.
Waiting before retry.
[02:28:01] + Attempting to get work packet
[02:28:01] - Connecting to assignment server
[02:28:02] - Successful: assigned to (171.64.65.60).
[02:28:02] + News From Folding@Home: Welcome to Folding@Home
[02:28:02] Loaded queue successfully.
[02:32:47] + Could not connect to Work Server
[02:32:47] - Attempt #5 to get work failed, and no other work to do.
Waiting before retry.
[02:34:10] + Attempting to get work packet
[02:34:10] - Connecting to assignment server
[02:34:11] - Successful: assigned to (171.64.65.60).
[02:34:11] + News From Folding@Home: Welcome to Folding@Home
[02:34:11] Loaded queue successfully.
[02:34:32] - Couldn't send HTTP request to server
[02:34:32] + Could not connect to Work Server
[02:34:32] - Attempt #6 to get work failed, and no other work to do.
Waiting before retry.

[k1wi]

I would think that there is a networking problem specific to you here.

Did anything change at your end around the time that you stopped being able to connect?

You could try temporarily shutting down your firewall and loaded the client to see whether that works. If it doesn't then you will probably need to have a look @ your router settings etc.

Add: could you give us some more information on your set up? For example what OS's are you running and how are you connecting to the net?

[bruce]

One simple test that might show something: Note that the Assignment Server thinks you can get work from 171.64.65.60. Can you open http://171.64.65.60 in your browser? I cannot, which is a bad sign. Serverstat is showing strange conditions for that server, too. I'll see if anybody at the Pande Group has any ideas.

[guest3412]

my client is successfully being assigned to 171.64.65.60 but the stats page shows reject??? I can't get the client to assign to a different work server... how do I tell it that server is down go to another server?

I can not open http://171.64.65.60 either.

Code: Select all

[03:35:45] + Attempting to send results [July 27 03:35:45 UTC]
[03:35:49] + Results successfully sent
[03:35:49] Thank you for your contribution to Folding@Home.
[03:35:49] + Number of Units Completed: 384

[03:35:53] - Preparing to get new work unit...
[03:35:53] + Attempting to get work packet
[03:35:53] - Connecting to assignment server
[03:35:54] - Successful: assigned to (171.64.65.60).
[03:35:54] + News From Folding@Home: Welcome to Folding@Home
[03:35:54] Loaded queue successfully.
[03:39:27] + Could not connect to Work Server
[03:39:27] - Attempt #1  to get work failed, and no other work to do.
Waiting before retry.
[03:39:45] + Attempting to get work packet
[03:39:45] - Connecting to assignment server
[03:39:46] - Successful: assigned to (171.64.65.60).
[03:39:46] + News From Folding@Home: Welcome to Folding@Home
[03:39:46] Loaded queue successfully.
[03:40:07] - Couldn't send HTTP request to server
[03:40:07] + Could not connect to Work Server
[03:40:07] - Attempt #2  to get work failed, and no other work to do.
Waiting before retry.
[03:40:27] + Attempting to get work packet
[03:40:27] - Connecting to assignment server
[03:40:28] - Successful: assigned to (171.64.65.60).
[03:40:28] + News From Folding@Home: Welcome to Folding@Home
[03:40:28] Loaded queue successfully.
[03:44:02] + Could not connect to Work Server
[03:44:02] - Attempt #3  to get work failed, and no other work to do.
Waiting before retry.
[03:44:32] + Attempting to get work packet
[03:44:32] - Connecting to assignment server
[03:44:32] - Successful: assigned to (171.64.65.60).
[03:44:32] + News From Folding@Home: Welcome to Folding@Home
[03:44:33] Loaded queue successfully.
[03:48:31] + Could not connect to Work Server
[03:48:31] - Attempt #4  to get work failed, and no other work to do.
Waiting before retry.
[03:49:18] + Attempting to get work packet
[03:49:18] - Connecting to assignment server
[03:49:19] - Successful: assigned to (171.64.65.60).
[03:49:19] + News From Folding@Home: Welcome to Folding@Home
[03:49:19] Loaded queue successfully.
[03:49:40] - Couldn't send HTTP request to server
[03:49:40] + Could not connect to Work Server
[03:49:40] - Attempt #5  to get work failed, and no other work to do.
Waiting before retry.
[03:51:01] + Attempting to get work packet
[03:51:01] - Connecting to assignment server
[03:51:02] - Successful: assigned to (171.64.65.60).
[03:51:02] + News From Folding@Home: Welcome to Folding@Home
[03:51:02] Loaded queue successfully.
[03:51:23] - Couldn't send HTTP request to server
[03:51:23] + Could not connect to Work Server
[03:51:23] - Attempt #6  to get work failed, and no other work to do.
Waiting before retry.
[03:54:18] + Attempting to get work packet
[03:54:18] - Connecting to assignment server
[03:54:18] - Successful: assigned to (171.64.65.60).
[03:54:18] + News From Folding@Home: Welcome to Folding@Home
[03:54:18] Loaded queue successfully.
[03:54:40] - Couldn't send HTTP request to server
[03:54:40] + Could not connect to Work Server
[03:54:40] - Attempt #7  to get work failed, and no other work to do.
Waiting before retry.
[04:00:02] + Attempting to get work packet
[04:00:02] - Connecting to assignment server
[04:00:03] - Successful: assigned to (171.64.65.60).
[04:00:03] + News From Folding@Home: Welcome to Folding@Home
[04:00:03] Loaded queue successfully.
[04:00:24] - Couldn't send HTTP request to server
[04:00:24] + Could not connect to Work Server
[04:00:24] - Attempt #8  to get work failed, and no other work to do.
Waiting before retry.

Shut down to try some setting changes.. still broken.....

[04:24:42] - Connecting to assignment server
[04:24:42] Connecting to http://assign.stanford.edu:8080/
[04:24:43] Posted data.
[04:24:43] Initial: 40AB; - Successful: assigned to (171.64.65.60).
[04:24:43] + News From Folding@Home: Welcome to Folding@Home
[04:24:43] Loaded queue successfully.
[04:24:43] Connecting to http://171.64.65.60:8080/
[04:26:43] Posted data.
[04:28:17] Initial: 003A; + Could not connect to Work Server
[04:28:17] - Attempt #1  to get work failed, and no other work to do.
Waiting before retry.
[04:28:25] + Attempting to get work packet
[04:28:25] - Will indicate memory of 2048 MB
[04:28:25] - Connecting to assignment server
[04:28:25] Connecting to http://assign.stanford.edu:8080/
[04:28:26] Posted data.
[04:28:26] Initial: 40AB; - Successful: assigned to (171.64.65.60).
[04:28:26] + News From Folding@Home: Welcome to Folding@Home
[04:28:26] Loaded queue successfully.
[04:28:26] Connecting to http://171.64.65.60:8080/
[04:30:26] Posted data.
[04:32:00] Initial: 003A; + Could not connect to Work Server
[04:32:00] - Attempt #2  to get work failed, and no other work to do.
Waiting before retry.


[bruce]

I said that server was doing strange things. It seems to be going back an forth between Rejecting connections and accepting them. It also has very high CPU Load and very high Net Load. A high Net Load is consistent with everybody being sent to that server, as you reported.

Are you configured for Small/Normal/Big WUs? Have you tried adding the -advmethods flag? Changes to either of those often gives the Assignment Server a reason to send you to another server, but there is no way to force it to send you somewhere else.

[toTOW]

lshurr> how much memory does your system reports ? what are the other settings ?

[guest3412]

I have 2 machines running the single clients. Both are having trouble with 171.64.65.60 it seems that they go with out work for hours sometimes, to finally get connected to the server and gain a WU. 1 machine has 2GB ram (2 clients) and the other 4GB (3 clients, and 1-480 GPU) both are able to use all the memory, the 2GB is a dual core (old 2ghz AMD, the 4GB is a quad (AMD 940 3ghz). scientific cores = yes and big packages - console client.

The client should be smarter I would think, so that when a server is overloaded it can request another assignment server. Especially when it errors out after several try's.

I still can NOT connect to http://171.64.65.60 and the stats page shows reject again!

[bruce]

The client should be smarter I would think, so that when a server is overloaded it can request another assignment server. Especially when it errors out after several try's.

I still can NOT connect to http://171.64.65.60 and the stats page shows reject again!

It's "Accepting" right now. Unfortunately the status still seems to be switching back and forth between Reject and Accepting. That means that any sort of decision is probably wrong a few minutes after the status is tested. The automatic software is doing the best it can, but this is a case where somebody will have to figure out what the underlying problem is and intervene.

By the way, it's not a matter of making the client smarter. The client isn't responsible for making that sort of decision. That's the job of the Assignment Server. Whenever the AS sees that this Work Server is Rejecting connections, it stops sending folks who want new WUs to that WS as long as there's another WS that has WUs that fit your requirements.

As far as your requirements are concerned, you're stuck with your OS and with your RAM size and the particular client that you're running (etc.), but adding/removing -advmethods sometimes changes your requirements just enough that the AS will redirect you to a different WS. Similarly, reconfiguring from Big to Normal to Small sometimes does it, too. It all depends on dynamic conditions, including which servers have WUs and the particular requirements of the Projects.

[lshurr]

I said that server was doing strange things. [etc...].

Are you configured for Small/Normal/Big WUs? Have you tried adding the -advmethods flag? Changes to either of those often gives the Assignment Server a reason to send you to another server, [etc...].

My thanks to everyone who responded, and especially to Bruce. Taking your advice, after upgrading client on both machines and finding that they continued to unsuccessfully request work from 171.64.65.60, I added -advmethods and both clients immediately obtained work units from 129.74.85.15. I would say that 171.64.65.60 is in trouble if I haven't been able to get work units since June 16th.

[guest3412]

So the AS is supposed to keep up with what server is up and down? ok so when my client asks the AS for a WU server, and the AS gives it IP (A), then seconds later I ask it for a WU server again. Why would the AS continue to say IP (A) with out asking server at IP (A) if it is accepting? I would think that if the AS is repeatability asked by user X at ip X and machine X over and over for a WU server, that the AS would look up IP (A) and say "hey are you accepting?" And when it doesn't reply, make a note that it's down and not repeatably send me there, on the other hand it could be as simple as adding to the client an ability to tell the AS that "I can't reach that WU server, please try another" it just seems to me that it shouldn't be that hard to note that when a server goes down, that the clients can just ask for a different WU server. But I'm not a programmer, just a data analyst. I see things in different ways sometimes.

[codysluder]

Nobody knows exactly how this works, but speaking as one who has done a moderate amount of programming I can see a couple of flaws in your suggestion.

First the AS doesn't keep track of who has asked it for an assignment or when. That would be a large amount of data to keep track of, and most of the time it would be useless. If I'm the information clerk in a sports stadium and somebody asks me where the restroom is, I judge whether they want the Men's room or the Women's room and direct them to where their needs will be met. This same process happens over and over. If somebody comes back and tells me that they need directions to a different one because that one is out of order, I'll gladly send them to a differnt one, but the client doesn't provide that information, so if they just come back and ask again, I'll (incorrectly) send them to the same place. Sooner or later I may learn that that restroom is out of order, and when I do, I'll send everybody to a different one. The only question here is how quickly I learn that the Men's server on Level 3-East is out of order or the Women's room on Level 2-West has a long line or there's no paper towels in Level 3-North.

Adding more complex software to the client is one possibility. Perhaps a better option would be for my employer to require me to check on the status of all of the restrooms in the stadium more frequently.

[guest3412]

i understand your point now, but i guess since the client needs to be overhauled and v7 is coming .. maybe they will implement the client telling the AS that "i can't talk to ip xx and need another WU server" this shouldn't be that hard to implement based upon your description.

[7im]

It doesn't do any good if a new client has a new feature to tell the AS something, if the AS isn't also upgraded to listen for the something. Clients and servers are often updated at the same time, but not always.