Too big units, comparing to just finished ones

Moderators: Site Moderators, FAHC Science Team

PawelS
Posts: 10
Joined: Fri Nov 08, 2019 2:23 pm

Too big units, comparing to just finished ones

Post by PawelS »

So from time to time I have a situation like ~60k Base Point unit is finished in 2.5 days and I get 80kBP with expiration in 2 days. I have a good GPU, but it is loud even at the medium setting, so I fold irregularly, but still giving a unit with a base credit/ time to expiration ratio that is bigger than a finished unit credit/ its time to finish ratio is wrong, especially that there is no option to reject such unit. I'm curious about the [s]unit dispatching algorithm[/s] how unit size is determined.
Last edited by PawelS on Mon Jul 12, 2021 6:39 am, edited 1 time in total.
gunnarre
Posts: 567
Joined: Sun May 24, 2020 7:23 pm
Location: Norway

Re: Too big units, comparing to just finished ones

Post by gunnarre »

Did you let the work unit run for about 5% before looking at the estimated time? The estimation become more accurate as the WU progresses.
That said, the assignment of WUs to GPUs could be better than it is at the moment, and there are plans for improving it - though I don't think there is a timeline for those assignment improvements to be implemented.
Image
Online: GTX 1660 Super, GTX 1080, GTX 1050 Ti 4G OC, RX580 + occasional CPU folding in the cold.
Offline: Radeon HD 7770, GTX 960, GTX 950
aetch
Posts: 447
Joined: Thu Jun 25, 2020 3:04 pm
Location: Between chair and keyboard

Re: Too big units, comparing to just finished ones

Post by aetch »

PawelS wrote:I have a good GPU, but it is loud even at the medium setting, so I fold irregularly, but still giving a unit with base credit/ time to expiration that is bigger than a finished unit credit/ its time to finish is wrong, especially that there is no option to reject such unit.
For the GPU the "Folding Power" slide bar only controls when your GPU folds, not how much effort it exerts. For that you need other tools.

What OS are you running?
There are things you can do to quieten your system down.

I'm going to assume you're running Windows
GPU - MSI Afterburner. It's a overclocking tool that also allows you to lower the GPU's power down to about 50-60% of the default power settings. Lowering the power limit also lowers your GPU temperature target and runs your fans slower/quieter because it doesn't have to expel as much heat.

Linux - can be done but it takes a lot more work to configure.
Folding Rigs - None (25-Jun-2022)

ImageImage
PawelS
Posts: 10
Joined: Fri Nov 08, 2019 2:23 pm

Re: Too big units, comparing to just finished ones

Post by PawelS »

Thanks for the answers. I'm not going to often switch GPU's power. I accept it is not a priority for developers, however, most people stop serviceless folding after a few units, so I guess noise and/or temperature could be a big factor because it makes ,,Start and it will not let you forget it" thing.
gunnarre wrote:Did you let the work unit run for about 5% before looking at the estimated time?
Most of the days it folds for maybe 2 hours, but on some much longer, so if I was fully dedicated expiration time would be very generous. Maybe it is some psychological pressure that works on some people.
gunnarre
Posts: 567
Joined: Sun May 24, 2020 7:23 pm
Location: Norway

Re: Too big units, comparing to just finished ones

Post by gunnarre »

Folding is time sensitive because the previous generation of simulations are used to set up the next one. In other words, folding is a relay race.

Covid Moonshot sprints also have a shorter than usual timeout and expiration. If you want to contribute to science just a couple hours per day, and there's nothing to do to improve consistency (like power limited GPU), then perhaps you might look into some projects from BOINC or World Community Grid which are less time-sensitive?
Image
Online: GTX 1660 Super, GTX 1080, GTX 1050 Ti 4G OC, RX580 + occasional CPU folding in the cold.
Offline: Radeon HD 7770, GTX 960, GTX 950
PawelS
Posts: 10
Joined: Fri Nov 08, 2019 2:23 pm

Re: Too big units, comparing to just finished ones

Post by PawelS »

When I was choosing a platform I liked FAH because of its openness about results. It's rare that my units are not completed on time and after this, I get easy units so today I got 47kBC with an expiration day in 5 days and it should be finished after 1 day, so FAH is also designed for less intensive and less systematic usage.
aetch
Posts: 447
Joined: Thu Jun 25, 2020 3:04 pm
Location: Between chair and keyboard

Re: Too big units, comparing to just finished ones

Post by aetch »

I try to fold 24/7. I'm currently just running the Ryzen system in my signature. I have the CPU clock limited to roughly 3.3GHz and the GPU power limited to 60% (~130W). Currently the GPU is running a work unit for project 18016 (base points 113K)which it picked up roughly 1/2 hour ago, it estimates it will be completed within the next 3.1/2 hours (about 4 hours from start to finish).

FAH benefits best from the the work units being completed and returned in a timely manner. The next unit going out depends on the previous unit having been completed and returned. Something I have suggested to others is to maybe run your system continuously for a couple of days a week (24-48 hour run) rather than doing a few hours a day where the contribution is more broken up.
Folding Rigs - None (25-Jun-2022)

ImageImage
PawelS
Posts: 10
Joined: Fri Nov 08, 2019 2:23 pm

Re: Too big units, comparing to just finished ones

Post by PawelS »

aetch wrote:I try to fold 24/7.
Thank you
I'm currently just running the Ryzen system in my signature. I have the CPU clock limited to roughly 3.3GHz and the GPU power limited to 60% (~130W).
I'll not be surprised if somebody suggests doing it at 100 %, at least.
FAH benefits best from the the work units being completed and returned in a timely manner.
That is why I started this topic.
The next unit going out depends on the previous unit having been completed and returned.
Sure, but because some units are returned closer to timeout or expiration date it doesn't mean others are going to wait idly to finish that unit because there are many other projects. Also sorting projects to users with similar efficiency would make sense.
Something I have suggested to others is to maybe run your system continuously for a couple of days a week (24-48 hour run) rather than doing a few hours a day where the contribution is more broken up.
I agree on the condition it is not problematic. If it is then better broken up contribution than none.
tvdsluis
Posts: 36
Joined: Mon Apr 24, 2017 11:12 am

Re: Too big units, comparing to just finished ones

Post by tvdsluis »

I see the same lately.
Up till now not much of a problem to get WU's finished well before timeout, i run mostly 16 hours a day at 100%.
Now i get WU's which are hard to finish before the timeout even when running 24 hours a day.
On one system with a 8 Gb Radeon RX 580 (i know an oldie but not that crappy i think) i have a WU now from project 18010.
That WU takes about 16.25 hours to finish.
Assigned 2021-07-13 time 14::::::::.44
Timeout 2021-07-14 time 08.14 + 1 (somehow can't get the number nine to print in this message, if i type the key nine i get 9)
So i have 18.5 hours to finish.
But that gpu needs 16.25 hours.
That's a very tight deadline. Expecting to return WU's 8 hours before the timeout to help science is impossible this way.
If the GPU is used in the mean time for something else, even with 24 hours a day, the timeout will be hard if not impossible to reach.
That means that my previous 16 hours a day is not good enough, only 24 hours a day will work, and even than it will always be a close call.
Most projects run fine but to it seems projects starting with 18 have this problem.
And i'm aware enough not to start a new WU if i know a computer will shutdown within an hour or so, so i press finish.
That helps a bit.
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: Too big units, comparing to just finished ones

Post by Neil-B »

The 18010 projects wus take a little over 3hrs with my rtx3070 so timeout looks OK for faster/wider gpus .. it may be that these should be restricted to just these and not be assigned to the slower/thinner (older) gpus to avoid issues with not meeting timeouts ?
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
tvdsluis
Posts: 36
Joined: Mon Apr 24, 2017 11:12 am

Re: Too big units, comparing to just finished ones

Post by tvdsluis »

Neil-B wrote:The 18010 projects wus take a little over 3hrs with my rtx3070 so timeout looks OK for faster/wider gpus .. it may be that these should be restricted to just these and not be assigned to the slower/thinner (older) gpus to avoid issues with not meeting timeouts ?
Yes, that is i think kind of my point.
If the science needs a tight timeout for certain projects, than that's a given.
But handing these out to GPU's that can hardly meet that time restraint, is a bit of a waste.
There are plenty of other projects that this GPU can easily finish in time, but it seems only to get projects in the 18*** range.
But maybe that's due to the fact that it's an AMD GPU (and thus non CUDA), and not much work other than this is available?
My NVIDIA's, also the older ones, have less of this problem it seems.
PeteHobbis
Posts: 11
Joined: Fri Feb 12, 2021 10:42 am
Location: uk

Re: Too big units, comparing to just finished ones

Post by PeteHobbis »

I'm a little fish in this pond. I only have my laptop, with the CPU plus an Nvidia GeForce MX250. Lately the GPU has been assigned mostly 18xxx units, of which three have been 18016. The first expired while I was at work; I was dismayed to have wasted two days' computing. Previously, units assigned have been well suited to the computer. I was surprised to get another unit of 18016 later; this one was on it's way to expiry, but I was there to see it, and it wasn't going to miss by much. I helped it past the line by sitting the computer on a freezer block! Then I paused the GPU so as not to get another unit of the same project; only I unpaused it without thinking a day or so later, and got another unit of 18016, which has just expired despite my freezer-block trick. So I've paused the GPU again, and I'll just run the CPU while this project's around.
gunnarre
Posts: 567
Joined: Sun May 24, 2020 7:23 pm
Location: Norway

Re: Too big units, comparing to just finished ones

Post by gunnarre »

On a laptop, you might be best served folding only on the GPU or only on the CPU. Even gaming laptops aren't thermally designed to run 100% CPU and 100% GPU at the same time - games are typically limited by one or the other. On my laptop, I'm only folding very lightly so I don't even hear the fans if the TV is on or the window is open.
Image
Online: GTX 1660 Super, GTX 1080, GTX 1050 Ti 4G OC, RX580 + occasional CPU folding in the cold.
Offline: Radeon HD 7770, GTX 960, GTX 950
BobWilliams757
Posts: 493
Joined: Fri Apr 03, 2020 2:22 pm
Hardware configuration: ASRock X370M PRO4
Ryzen 2400G APU
16 GB DDR4-3200
MSI GTX 1660 Super Gaming X

Re: Too big units, comparing to just finished ones

Post by BobWilliams757 »

There does seem to be a larger quantity of big/long/wide WU's lately that are hitting the lower end gear.

I've probably had more of those type WU's in the last month or two than I've had in the last 15 months of folding. I don't really mind that my system won't make a deadline, but it's not good for the science when I know I pick up a WU that will probably be reissued and completed before my rig finishes it up. I always just let them run hoping at least the one I'm running will be done before the other one issued, but that's not the case sometimes.

If I have work running, I fold 24/7. But I dislike knowing that time might be essentially wasted, and dislike even more that trying to help might slow things down.

FWIW I wouldn't mind if I had less WU's that my system could pick up if it helps the work get done quicker. I'm sure there is plenty of work for the slower systems if the restrictions are tightened up some. And if not..... better to let the faster systems do it anyway.
Fold them if you get them!
Trokari
Posts: 2
Joined: Tue Dec 07, 2021 9:54 pm

Re: Too big units, comparing to just finished ones

Post by Trokari »

How about being able to dynamically distribute the required simulations into smaller packages? I don't know at all if it's possible to implement in FAH's case, tho. It would be nice if you could choose the WU size you prefer from the client's UI
and the work server would then be able to split the calculations into requested size and go from there.

There could be certain, fixed work unit sizes that folders would be able to choose from. This wouldn't cause problem with the WU time sensitivity either and actually it would probably improve on getting calculations finished in
time as fewer work units would go to waste due to expiration at least for intermittent folders like myself.
Post Reply