Delay in points for completed units

[AJMSmith]

It appears that the allocation of points for returned units is being delayed for one reason or another ATM. I'm finding that a unit completed at say 14:27Z is not included in my points total until the update at 15:05Z ie after the next update of the 3rd party files. If this is a persistent problem it might be an idea to bring the hourly update forward by say 10 minutes so that they 3rd party stats file is more current than it would otherwise be.

[bruce]

It's unlikely that PG will respond to this suggestion unless it becomes a popular suggestion on their new communications site:
https://www.reddit.com/r/foldingathome

[AJMSmith]

What I was suggesting was that it might be a bug in the system but that I'm not sure and if it was I thought that my suggestion might be a fix.

[gwildperson]

"It's not a bug; it's a feature."

Three things have to happen. All WUs completed since the last update must be collected and added to the official database. Data from that database must be exported to the flat files. The 3rd party sites must download those flat files and process them.

These steps must be completed in sequence, so if step 1 takes longer than usual, incomplete data will arrive at the 3rd party sites. Unfortunately, sometimes step 1 takes significantly longer than expected, such as when that process has been suspended for a while, so a safety factor has to be included since those 3 steps happen asynchronously.

[AJMSmith]

I'm fully aware of the sequence of events it's just that the timing could be better. For some time now it appears that to be included in an update to the 3rd party files the unit needs to be completed at leas an hour before the update and sometimes 2 hours.

BTW why can't one process update the timestamp on a given file when it completes and the next process can use that as a flag saying that it's safe to proceed but if run before the flag is set then it suspends itself for a short while and tests the flag again ... that would remove the need for a safety factor.

[billford]

FAH starts its stats run every hour on the hour, the summary file is usually complete and available in full about 15-20 minutes later. It's up to the 3rd party stats sites when they download it.

As far as I can ascertain (from observation, I've no inside knowledge):

Kakao tries to get it as soon as possible, this means that if something causes the stats run to take longer than usual then it may only get partial data. The rest will be picked up next time (ie 3 hours later).

EOC leaves it until almost the top of the hour so it's unlikely to miss anything, but means it's data is (apparently) an hour behind.

I suspect that what you're seeing is an effect of the current tendency of the stats run to occasionally crash partway through, so a WU that wasn't incorporated before the crash has to wait until the next run at the top of the next hour. And may have to wait another 3 hours before being picked up by a 3rd party site.

PG are aware of the problem and have put in a temporary fix- viewtopic.php?f=18&t=26191&start=195#p270724 - but the point is that if there's a timing problem it's up to the 3rd party site to fix it at their end.

[7im]

This is how it has always worked. This isn't the 30 minutes or less pizza place. It has always taken up to an hour (and some times longer) after receiving a work unit to get credited. And there was never any guarantee as to when it would hit the posted stats. It could be the current 3 hour cycle or the next, depending on how recently the work servers synced up with the stats server. It's unrealistic to expect all work servers to try to upload all their points info at 2:59 for the 3:00 update. The uploads are spaced out intervals. Your example just missed that interval and got picked up on the next one.

[Nathan_P]

Points are allocated based on return time to the server so even if your points don't show up when you expect you will have received proper credit, you can always check your stats at http://folding.stanford.edu/home/teams-stats/ , Stats for the 3rd party sites will always be behind - with Kakao shutting down soon that will only leave EOC as a 3rd party site - as these are run privately and rely on donations and the owners free time to make upgrades it may take awhile to get any fixes implemented.

Note that a server glitch at PG could also cause a delay in points showing up.