Low on work units?

Moderators: Site Moderators, FAHC Science Team

mwroggenbuck
Posts: 127
Joined: Tue Mar 24, 2020 12:47 pm

Low on work units?

Post by mwroggenbuck »

Are we getting low in CPU work units? I have had a couple of times in the past few days where my system took a couple of hours to download a work unit. Is there something wrong with my system, or is there a known issue with the distribution complex?
gunnarre
Posts: 567
Joined: Sun May 24, 2020 7:23 pm
Location: Norway

Re: Low on work units?

Post by gunnarre »

I'm seeing this too. It's not just your system. One of my machines was waiting for about 40 minutes for a CPU unit yesterday. Here's what it looks like today:

Code: Select all

10:33:08:WU01:FS00:Connecting to assign1.foldingathome.org:80
10:33:09:WARNING:WU01:FS00:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
10:33:09:WU01:FS00:Connecting to assign2.foldingathome.org:80
10:33:10:WARNING:WU01:FS00:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
10:33:10:WU01:FS00:Connecting to assign3.foldingathome.org:80
10:33:10:WARNING:WU01:FS00:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
10:33:10:WU01:FS00:Connecting to assign4.foldingathome.org:80
10:33:11:WARNING:WU01:FS00:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
10:33:11:ERROR:WU01:FS00:Exception: Could not get an assignment
10:33:11:WU01:FS00:Connecting to assign1.foldingathome.org:80
10:33:12:WARNING:WU01:FS00:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
10:33:12:WU01:FS00:Connecting to assign2.foldingathome.org:80
10:33:12:WARNING:WU01:FS00:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
10:33:12:WU01:FS00:Connecting to assign3.foldingathome.org:80
10:33:13:WARNING:WU01:FS00:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
10:33:13:WU01:FS00:Connecting to assign4.foldingathome.org:80
10:33:14:WARNING:WU01:FS00:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
10:33:14:ERROR:WU01:FS00:Exception: Could not get an assignment
10:34:11:WU01:FS00:Connecting to assign1.foldingathome.org:80
10:34:12:WARNING:WU01:FS00:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
10:34:12:WU01:FS00:Connecting to assign2.foldingathome.org:80
10:34:13:WARNING:WU01:FS00:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
10:34:13:WU01:FS00:Connecting to assign3.foldingathome.org:80
10:34:13:WARNING:WU01:FS00:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
10:34:13:WU01:FS00:Connecting to assign4.foldingathome.org:80
10:34:14:WARNING:WU01:FS00:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
10:34:14:ERROR:WU01:FS00:Exception: Could not get an assignment
10:34:33:WU02:FS00:0xa8:Completed 480000 out of 500000 steps (96%)
10:35:49:WU01:FS00:Connecting to assign1.foldingathome.org:80
10:35:49:WU01:FS00:Assigned to work server 128.252.203.9
10:35:49:WU01:FS00:Requesting new work unit for slot 00: RUNNING cpu:8 from 128.252.203.9
10:35:49:WU01:FS00:Connecting to 128.252.203.9:8080
10:35:50:WU01:FS00:Downloading 8.14MiB
10:35:56:WU01:FS00:Download 88.32%
10:35:57:WU01:FS00:Download complete
10:35:57:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:13827 run:702 clone:4 gen:170 core:0xa7 unit:0x000000c380fccb095e6d30e64a016124
Image
Online: GTX 1660 Super, GTX 1080, GTX 1050 Ti 4G OC, RX580 + occasional CPU folding in the cold.
Offline: Radeon HD 7770, GTX 960, GTX 950
JimF
Posts: 652
Joined: Thu Jan 21, 2010 2:03 pm

Re: Low on work units?

Post by JimF »

Apparently so.
viewtopic.php?f=18&t=36114#p343124

You would not know it from the project stats.
https://apps.foldingathome.org/serverstats

I am just doing GPU work now, and it is fine.
comixgoddess
Posts: 88
Joined: Wed Apr 08, 2020 9:57 pm
Location: Pacific Northwest

Re: Low on work units?

Post by comixgoddess »

I can only do CPU folding, and my wait time for a new work unit has varied between seconds and almost an hour.
Image
PantherX
Site Moderator
Posts: 7020
Joined: Wed Dec 23, 2009 9:33 am
Hardware configuration: V7.6.21 -> Multi-purpose 24/7
Windows 10 64-bit
CPU:2/3/4/6 -> Intel i7-6700K
GPU:1 -> Nvidia GTX 1080 Ti
§
Retired:
2x Nvidia GTX 1070
Nvidia GTX 675M
Nvidia GTX 660 Ti
Nvidia GTX 650 SC
Nvidia GTX 260 896 MB SOC
Nvidia 9600GT 1 GB OC
Nvidia 9500M GS
Nvidia 8800GTS 320 MB

Intel Core i7-860
Intel Core i7-3840QM
Intel i3-3240
Intel Core 2 Duo E8200
Intel Core 2 Duo E6550
Intel Core 2 Duo T8300
Intel Pentium E5500
Intel Pentium E5400
Location: Land Of The Long White Cloud
Contact:

Re: Low on work units?

Post by PantherX »

Please note that we have some CPU Projects in the pipeline that will be addressing this shortage very soon :)

If you have any system maintenance/administration tasks, now would be a good time to do it :)
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time

Welcome To The F@H Support Forum Ӂ Troubleshooting Bad WUs Ӂ Troubleshooting Server Connectivity Issues
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Low on work units?

Post by bruce »

It should be noted that research requires a lot of scientific thought. Developing a new project, doesn't happen instantaneously and once somebody has a plan, there's a lot of setup work involved so as weknow that new projects are almost ready to release, there may be hiccups in the process, but they won't be long ones. It's not as simple as just adding another level to game software.

Actually, this a a good problem to have. It means that hardware, both at the client level and at the server level, has finally caught up with the initial COVID surge.
JimF
Posts: 652
Joined: Thu Jan 21, 2010 2:03 pm

Re: Low on work units?

Post by JimF »

I am happy that the work is going well. But I don't know of any server status page that tells us what is going on.
We don't know if there is a server problem or a work shortage. We shouldn't have to rely on a moderator to check it.
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: Low on work units?

Post by Neil-B »

The public server stats page shows the lack of WSs giving out A8 or A7 ... the large number of both are all on the servers that are on accept only ... gives a fairly clear indication of low availability tbh
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
JimF
Posts: 652
Joined: Thu Jan 21, 2010 2:03 pm

Re: Low on work units?

Post by JimF »

If you look at the Project Type Stats, it appears that everything is OK. Beyond that, I think you need to guess.
The guess is of course helped by the fact that you can't get any work. But it is not much of a guess then.
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Low on work units?

Post by bruce »

Several projects are being moved from older servers to a new server. I suspect that's interfering with the client<>server transfers and it'll be resolved soon. Also, some new projects are making their way through the beta testing process.

There's hope on both fronts.
JimF
Posts: 652
Joined: Thu Jan 21, 2010 2:03 pm

Re: Low on work units?

Post by JimF »

Thanks, I know it is improving.
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: Low on work units?

Post by Neil-B »

If you look at the individual server rows you can see how the headline project type stats are made up ... most of the cpu a7 and A8 are listed in accept only or down servers so they are not being assigned and are actually available at the moment ... this isnt a guess it is what the data in the table is showing :) ... as to stating a reason for the servers being in accept only or down that would for me be a guess but the big cluster of similarly name servers is new and I would infer this is a new group of servers being commissioned and not yet fully up and stable - but yes that would be a guess :)
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
JimF
Posts: 652
Joined: Thu Jan 21, 2010 2:03 pm

Re: Low on work units?

Post by JimF »

Neil-B wrote:... as to stating a reason for the servers being in accept only or down that would for me be a guess but the big cluster of similarly name servers is new and I would infer this is a new group of servers being commissioned and not yet fully up and stable - but yes that would be a guess :)
All true, but you have just been told that a new group of servers are being added. That is not generally the information you have when you check the status.
On almost all BOINC projects (except for WCG, which does its own thing), they have a more useful server status which does not require a priori knowledge of what is going on.

Now Folding is different, in that it has always has work available and you really did not need to check that. But that was before COVID. We live in a new world.
mwroggenbuck
Posts: 127
Joined: Tue Mar 24, 2020 12:47 pm

Re: Low on work units?

Post by mwroggenbuck »

From what I can tell, this problem has effectively gone away. My new WU download now happen on the first or second try.

I hope others are seeing the same.
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: Low on work units?

Post by Neil-B »

JimF wrote:
Neil-B wrote:... as to stating a reason for the servers being in accept only or down that would for me be a guess but the big cluster of similarly name servers is new and I would infer this is a new group of servers being commissioned and not yet fully up and stable - but yes that would be a guess :)
All true, but you have just been told that a new group of servers are being added. That is not generally the information you have when you check the status.
Actually I infer it based on the cluster of names and that they are not ones I have seen until recently ... it didn't need anyone to tell me anything tbh
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
Post Reply