Problems downloading statistics files?

Moderators: Site Moderators, FAHC Science Team

Problems downloading statistics files?

Postby agrikk » Mon Aug 17, 2020 4:46 pm

In the last few days I noticed that my regular download of files from apps.foldingathome.org have been failing or taking an incredibly long time to complete.

https://apps.foldingathome.org/daily_us ... ry.txt.bz2
https://apps.foldingathome.org/daily_te ... ry.txt.bz2

normally take a few seconds to download, but now it's taking upwards of twenty minutes with multiple retries and connection closures. I'm on the end of a ten gigabyte pipe and I'm getting 44kB/s download speeds.

Is there something going on that I should know about? Is this a temporary phenomenon?
agrikk
 
Posts: 5
Joined: Wed Dec 18, 2013 9:57 pm

Re: Problems downloading statistics files?

Postby PantherX » Mon Aug 17, 2020 8:25 pm

Welcome to the F@H Forum agrikk,

I have asked the people responsible about this so let's see what happens. I am aware of another report about the Individual Donor Stats not loading (viewtopic.php?f=16&t=35987).
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time

Welcome To The F@H Support Forum Ӂ Troubleshooting Bad WUs Ӂ Troubleshooting Server Connectivity Issues
User avatar
PantherX
Site Moderator
 
Posts: 6733
Joined: Wed Dec 23, 2009 10:33 am
Location: Land Of The Long White Cloud

Re: Problems downloading statistics files?

Postby UofM.MartinK » Mon Aug 17, 2020 8:29 pm

If many people need these files regularly, and the front-end is not easy to scale up for whatever reason, would be publishing the stats as torrent a potential solution?

This could even be community-driven, if there are no legal implications against it...
UofM.MartinK
 
Posts: 55
Joined: Tue Apr 07, 2020 9:53 pm

Re: Problems downloading statistics files?

Postby PantherX » Mon Aug 17, 2020 9:29 pm

The update is that everything seems fine from the back-end. If it is a routing or a network issue, there's almost nothing that F@H can do about it.
User avatar
PantherX
Site Moderator
 
Posts: 6733
Joined: Wed Dec 23, 2009 10:33 am
Location: Land Of The Long White Cloud

Re: Problems downloading statistics files?

Postby agrikk » Mon Aug 17, 2020 9:33 pm

PantherX wrote:Welcome to the F@H Forum agrikk,

I have asked the people responsible about this so let's see what happens. I am aware of another report about the Individual Donor Stats not loading (viewtopic.php?f=16&t=35987).


Thank you! :)

I've been pulling these stats files since 2013 and since covid, I understand that availability (both in data files and actual work units) has been taxed.
agrikk
 
Posts: 5
Joined: Wed Dec 18, 2013 9:57 pm

Re: Problems downloading statistics files?

Postby UofM.MartinK » Mon Aug 17, 2020 9:51 pm

PantherX wrote:The update is that everything seems fine from the back-end. If it is a routing or a network issue, there's almost nothing that F@H can do about it.


Other than (also) distributing it as torrent? :)

Each file with date tag - why not? Is it worth an experiment?

Update: add a fancy way of signing to guarantee authenticity... perhaps even as part of the filename? :)
UofM.MartinK
 
Posts: 55
Joined: Tue Apr 07, 2020 9:53 pm

Re: Problems downloading statistics files?

Postby PantherX » Tue Aug 18, 2020 8:31 am

UofM.MartinK wrote:...Other than (also) distributing it as torrent? :)

Each file with date tag - why not? Is it worth an experiment?

Update: add a fancy way of signing to guarantee authenticity... perhaps even as part of the filename? :)

I didn't mention the feature during my conversation. However, you're more than welcomed to submit your ideas here (assuming that it hasn't already been suggested before): https://github.com/FoldingAtHome/fah-issues/issues
User avatar
PantherX
Site Moderator
 
Posts: 6733
Joined: Wed Dec 23, 2009 10:33 am
Location: Land Of The Long White Cloud

Re: Problems downloading statistics files?

Postby UofM.MartinK » Tue Aug 18, 2020 4:53 pm

Doesn't seem to be in high demand, based on the reaction here, and the files are smaller than I thought :)

If bandwidth or availability at various locations would be a problem regularly, once could also distribute them with a CDN like cloudflare.

But that doesn't seem to be a case, do no need to fix something that ain't broken :)
UofM.MartinK
 
Posts: 55
Joined: Tue Apr 07, 2020 9:53 pm

Re: Problems downloading statistics files?

Postby Neil-B » Tue Aug 18, 2020 5:30 pm

phew ... the torrent elephant in the room seems to have wandered off of its own accord ... relaxes, smiles, heart rate lowers ... :D
1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent, Quadro K420 1GB, FAH 7.6.13
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro, Quadro M1000M 2GB, FAH 7.6.13
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro, GTX 750Ti 2GB, FAH 7.6.13
Neil-B
 
Posts: 1405
Joined: Sun Mar 22, 2020 6:52 pm
Location: UK


Return to Discussions of General-FAH topics

Who is online

Users browsing this forum: No registered users and 1 guest

cron