Page 14 of 21

Re: Stats not updating?

Posted: Mon Sep 11, 2017 7:46 am
by Luscious
I really don't want to beat on a dead horse here but this is the 3rd interruption in as many weeks. Server infrastructure failing at that rate should be a red flag. Can we please get a long-term fix???

Re: Stats not updating?

Posted: Mon Sep 11, 2017 12:16 pm
by kookykrazee
Fortunately, folding is still working, just stats are not updating :)

Re: Stats not updating?

Posted: Mon Sep 11, 2017 1:50 pm
by JimboPalmer
Luscious wrote:I really don't want to beat on a dead horse here but this is the 3rd interruption in as many weeks. Server infrastructure failing at that rate should be a red flag. Can we please get a long-term fix???
I feel certain that if you fund it specifically, it will happen. I suspect that the Pande Groups funds are all tied up in studying proteins.

or

We do stuff for them, they do not have that priority toward us. The purpose of F@H is not the donors, it is the results.

Re: Stats not updating?

Posted: Mon Sep 11, 2017 2:27 pm
by Vig
JimboPalmer wrote: We do stuff for them, they do not have that priority toward us. The purpose of F@H is not the donors, it is the results.
While the purpose is the results, they come from the donors. You can't ignore the producers and still expect the production to continue.

Re: Stats not updating?

Posted: Mon Sep 11, 2017 2:34 pm
by drougnor
A single process on a single server having issues is not evidence of 'server infrastructure failing'. All this is is an hourly query against the existing database for the most recent set of numbers (Points total, Work Units total) for each entity (Donor or Team) that gets stuffed into a single file for digestion by whatever means third party users put it to.

It's not doom and gloom for the project if this process keeps failing. It's merely annoying for those of us who are collecting the data for our own usage, and for those of us who have grown USED to the pretty presentation of that data in easy to digest graphs.

Yes, the data presented by the third party sites is beneficial for drawing new folks in as a means of providing new goals to work towards and gamifying the F@H experience. But let's not pretend that it's the End of Days for the project.

I only hope a solution is found sooner than later and IF that solution is to create another means for 3rd parties like me to get the stats, that it's easy to slot into my existing solution without having to retool a bunch of old code.

Until then, I'm happily folding along and will keep an eye on my stats system to see if the errors have stopped and will gladly return here to report any new errors that come along that AREN'T caused by my own system.

d

Re: Stats not updating?

Posted: Mon Sep 11, 2017 5:11 pm
by Simplex0
"A single process on a single server having issues is not evidence of 'server infrastructure failing'. "

Yes. Providing that this IS only a problem with a single server that occurs 1 - 2 times every week an have been going on for more than a year now and not the only problem we observe of a much bigger problem.

At least this problem that has been going on for more than a year now makes you wonder if they have enough money to run this project properly.

Re: Stats not updating?

Posted: Mon Sep 11, 2017 6:14 pm
by Joe_H
The stats database has been restarted. It may take a couple hours to catch up on the backlog of WU's waiting to be entered into it.

As has been posted before, they are in the process of moving to new equipment for the stats database and front end processing.

Re: Stats not updating?

Posted: Mon Sep 11, 2017 6:29 pm
by drougnor
I can verify that the 2pm Eastern update posted and was pulled by my system. We're looking good for EOC to be able to post their first new numbers on their 3pm Central update.

Re: Stats not updating?

Posted: Mon Sep 11, 2017 8:48 pm
by WhitehawkEQ
And as soon as it's up, it's down again. :x
Here's a thought, How about a backup stat server to take over when main stat server goes down?

Re: Stats not updating?

Posted: Mon Sep 11, 2017 9:18 pm
by JimboPalmer
WhitehawkEQ wrote:And as soon as it's up, it's down again. :x
Here's a thought, How about a backup stat server to take over when main stat server goes down?
All it takes is your funding. If Pande Group had more funding, they would use it to do more Biochemistry.

Re: Stats not updating?

Posted: Mon Sep 11, 2017 9:19 pm
by bruce
Syncing a main and a backup copy of the database would be problematic. Are you suggesting that in the event of a problem, the stats queries might revert to a copy that's an additional hour old? How would the 3rd party stats people deal with a flat-file that is earlier than the last one they downloaded?

Re: Stats not updating?

Posted: Mon Sep 11, 2017 9:25 pm
by SteveWillis
bruce wrote:Syncing a main and a backup copy of the database would be problematic. Are you suggesting that in the event of a problem, the stats queries might revert to a copy that's an additional hour old? How would the 3rd party stats people deal with a flat-file that is earlier than the last one they downloaded?
Mirroring. Fail over should be seamless. Expensive though, I'd imagine. But hey, I don't really know enough about it to have an opinion. :-)

Re: Stats not updating?

Posted: Mon Sep 11, 2017 9:25 pm
by Vig
I believe he's talking about replication as opposed to one "coming up" only when the other one fails. Both servers pull stats from the database at the same time. Server 1 is usually used unless the system detects that it is down at which point Server 2's copy is used.

If the issue isn't with the servers, but instead with the database then that's not a viable option, but that doesn't seem like it is the case.

Re: Stats not updating?

Posted: Mon Sep 11, 2017 9:29 pm
by SteveWillis
Anyway, I see it's back on line.

Re: Stats not updating?

Posted: Mon Sep 11, 2017 10:30 pm
by bruce
Vig wrote:If the issue isn't with the servers, but instead with the database then that's not a viable option, but that doesn't seem like it is the case.
To me, that's unknown. Apparently the server recovers when rebooted. They're "old" servers and there's a plan to move the processing off of them.

Nevertheless, most of FAH's support is provided by students and the academic calendar indicates that we're between terms, so virtually everybody has left campus until the next term starts.