Logs and databases

If you're new to FAH and need help getting started or you have very basic questions, start here.

Moderators: Site Moderators, FAHC Science Team

Logs and databases

Postby FoldingFodder » Thu Apr 23, 2020 11:52 pm

Hi,

Are the assigned, timeout, and expiration epoch timestamps stored long term?

I see they're temporarily stored in the Units table of the client database whilst the core/client is working on the WU.

I'm surprised not seeing them in the Projects table since it has some history. Also surprised to see there is nothing stored in the description column, not even a generated link to a URI.

I've looked in databases, logs, xmls, jsons but can't find anything long term. So i'm wondering if i've missed the information or if it's else where.
FoldingFodder
 
Posts: 44
Joined: Sat Apr 04, 2020 7:07 pm

Re: Logs and databases

Postby Neil-B » Fri Apr 24, 2020 12:16 am

Assigned, returned and credited are all stored server side for every WU result from every folder ... the timeout and expiration periods are recorded server side for every project as is the server information ... not sure why this would be specifically stored elsewhere - though certain elements of this are stored in logs and for people using either their own logging scripts (I scrape my logs for every WU PRCG and pull the info from WU stats to give me precise points counts and timings) or tools such as HFM this data may be historically cached client side by these.

Would some of the info in the projects table be there simply to help the client publish ETA figures? ... as such it wouldn't need to store links to projects descriptions just some minimal completion type info?
1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent, Quadro K420 1GB, FAH 7.6.13
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro, Quadro M1000M 2GB, FAH 7.6.13
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro, GTX 750Ti 2GB, FAH 7.6.13
Neil-B
 
Posts: 1351
Joined: Sun Mar 22, 2020 6:52 pm
Location: UK

Re: Logs and databases

Postby FoldingFodder » Fri Apr 24, 2020 12:41 am

Neil-B wrote:Assigned, returned and credited are all stored server side for every WU result from every folder ... the timeout and expiration periods are recorded server side for every project as is the server information ... not sure why this would be specifically stored elsewhere - though certain elements of this are stored in logs and for people using either their own logging scripts (I scrape my logs for every WU PRCG and pull the info from WU stats to give me precise points counts and timings) or tools such as HFM this data may be historically cached client side by these.


The client seems to store everything else other than these long term - some information is seen in multiple logs and the databases. Which is why it made me wonder if it was stored else where or whether i had not spotted it.

Neil-B wrote:Would some of the info in the projects table be there simply to help the client publish ETA figures? ... as such it wouldn't need to store links to projects descriptions just some minimal completion type info?


There is a column specifically called description, which every entry is empty. The projects table is fairly bare with only a few columns.


My plan is similar to what you do. Collect what i want locally and then the remaining information from the servers.
FoldingFodder
 
Posts: 44
Joined: Sat Apr 04, 2020 7:07 pm

Re: Logs and databases

Postby Neil-B » Fri Apr 24, 2020 12:58 am

I made sure I changed the logs to keep more than the last 16 - I set log-rotate-max to 99 (wasn't sure anything over this works) and lift them off the box.

Some fairly simple parsing of a log grep allowed me to create the URLs for the WUs I have folded and grab the information from the WU stats ... There are bits I could do better - I use my version of "estimated PPD" by using the days figure from the WU stats (which includes server side and upload delays - luckily I tend to have few of those) whereas I could parse out the true "folding time" for every WU and match that with the actual points awarded to get a much truer figure.

Most of the time I am really doing the stats just o keep my mind working a bit whilst in lockdown - they are reasonably useful for knowing if beta projects I am running are slower/quicker than norm and awarding a kind of expected amount of points ... As data goes I am happy to grab what is easy and find a way of creating the rest I feel I need :)
Neil-B
 
Posts: 1351
Joined: Sun Mar 22, 2020 6:52 pm
Location: UK


Return to New Donors start here

Who is online

Users browsing this forum: Yandex [Bot] and 1 guest

cron