Langouste -- WU upload/download de-coupler (+upload capping)

This forum contains information about 3rd party applications which may be of use to those who run the FAH client and one place where you might be able to get help when using one of those apps.

Moderator: Site Moderators

rickoic
Posts: 322
Joined: Sat May 23, 2009 4:49 pm
Hardware configuration: eVga x299 DARK 2070 Super, eVGA 2080, eVga 1070, eVga 2080 Super
MSI x399 eVga 2080, eVga 1070, eVga 1070, GT970
Location: Mississippi near Memphis, Tn

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by rickoic »

On a slightly different subject.

Does anyone know what the FAH client is doing from the time that it says its writing core data to disk and takes about 6 seconds to say done, but then (at least for me) takes just over an hour before it actually begins to transmitt?

Tks
Rick
I'm folding because Dec 2005 I had radical prostate surgery.
Lost brother to spinal cancer, brother-in-law to prostate cancer.
Several 1st cousins lost and a few who have survived.
tear
Posts: 254
Joined: Sun Dec 02, 2007 4:08 am
Hardware configuration: None
Location: Rocky Mountains

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by tear »

Here are the details of this issue.

tear
One man's ceiling is another man's floor.
Image
whynot
Posts: 91
Joined: Wed Mar 26, 2008 9:02 pm
Location: Kyiv, Ukraine

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by whynot »

tear wrote:Every Linux distribution has a software/package management system (you use it
to install new software, remove unused software and so on). I don't know how Ubuntu's
tool called but you should able to find (point-and-click) it. Then it's just matter of
locating/searching for "gcc", "make" and installing them.
From what I'd observed (I maybe ignorant) KDE and GNOME come with different APT-frontends. While 'synaptics' (or such) is somewhat xdm-agnostic (if it comes pre-installed). Thus, I think, a route should be Start (or what?) -> System Administration -> Package Manegement.
tear wrote:As Ubuntu's based on Debian you might also try (as an alternative to previous paragraph)
running these commands in the terminal:

Code: Select all

apt-get install gcc
apt-get install make
I haven't used Debian in a long, long time so I might be slightly off here.
I'm sure other folks could be of more help. Guys?
I've just checked what would happen if I unistall 'libc6-dev'. It would take away 'build-essential', couple of '*-dev' and 'g++'. 'gcc' stays. I doubt that anything could be built without 'libc6-dev' (btw, I've just checked 'gcc' doesn't depend on 'libc6-dev' eather). Thus that should be:

Code: Select all

sudo apt-get install build-essential
That adds 'g++' (what brings 'gcc'), 'make', and 'libc6-dev'. And, I think, that should be explicitly noted that this must be issued as root (those who know about root know how to build).
tear wrote:As to why it doesn't come precompiled -- to me it's just not... kosher. Many things
affect binary compatibility and certain risks exist when providing precompiled binaries
If you ever go providing binaries then bubuntu would be enough. Anyone else has clue.

CORRECTION: I've investigated a bit and found that 'gcc' 'Recommends:' 'libc6-dev'. Since bubuntu has install-recommends insanity on by default 'apt-get install gcc' will install 'libc6-dev' too.
--
I'm counting for science.
Points just make me sick.
linuxfah
Posts: 8
Joined: Tue Sep 22, 2009 9:28 pm

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by linuxfah »

Hello tear

A couple people have asked me to include a binary of your Langouste software with the virtual Linux image that is being used to fold bigadv in Windows via VMware. I was wondering if it is okay that I include a binary of your program with this Linux image? I have been able to successfully compile your source. Thanks.
tear
Posts: 254
Joined: Sun Dec 02, 2007 4:08 am
Hardware configuration: None
Location: Rocky Mountains

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by tear »

Hey LF,

IANAL but if you haven't modified the code I'd say "go for it".

Ah, and let know of any problems you find; thanks!


tear

P.S.
On a holiday break now so pardon me for late response
One man's ceiling is another man's floor.
Image
linuxfah
Posts: 8
Joined: Tue Sep 22, 2009 9:28 pm

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by linuxfah »

Thanks tear. I did not modify the code. I only compiled it (make dep; make), which worked without issues.
tear
Posts: 254
Joined: Sun Dec 02, 2007 4:08 am
Hardware configuration: None
Location: Rocky Mountains

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by tear »

Maintenance release is now available at http://darkswarm.org/langouste3-0.12.2.tar.gz
I'll update first post momentarily. Excerpt from CHANGES file follows.

2010-01-01 -- New release (langouste3-0.12.2)
-- maintenance release; upgrade if you're experiencing issues related
 to items below
-- adds support for SLES 10 (SP2); it uses a different format of socket
 entries in /proc/pid/fd (thanks go to bingo-dog for troubleshooting and
 initial fix)
-- resolves a bug in socket lookup code that could result (under
 certain circumstances) in langouste not being able to find FAH
 client's PID; this bug has not been seen in the wild
 (thanks go to bingo-dog for identification and initial fix)
-- [advanced users] lazy signals finally work as expected
-- corrects a typo in an error message (found by bingo-dog)
-- tunes up the Makefile
-- adds clarifications to README (langouste needs to be run as the
 same user as FAH client(s))


tear
One man's ceiling is another man's floor.
Image
mattifolder

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by mattifolder »

Hello,

I've a feature requests:
- is it possible to append the send entries of the log of the forked client in the tmp-directory to FAHlog.txt in clients directory:
[09:03:54] Attempting to return result(s) to server...
[09:03:54] Project: 2671 (Run 14, Clone 42, Gen 181)
[09:03:54] - Read packet limit of 540015616... Set to 524286976.


[09:03:54] + Attempting to send results [January 9 09:03:54 UTC]
[09:09:48] pleted 2500 out of 250000 steps (1%)
[09:14:44] + Results successfully sent
[09:14:44] Thank you for your contribution to Folding@Home.
[09:14:44] + Number of Units Completed: 23
- can you migrate the "local"-entry of client.cfg of the forked client in the tmp-directory to client.cfg in clients directory:
local=23
Is the way to do that to change langouste-helper.sh ? If that is the right point, I'll attempt it by myself.

greatings mattifolder
tear
Posts: 254
Joined: Sun Dec 02, 2007 4:08 am
Hardware configuration: None
Location: Rocky Mountains

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by tear »

Hi Matti, Welcome to the forum,

Re first item
I'm not sure it's possible. I encourage you to experiment a little bit and yes, helper script
is most likely the best spot.

The risk comes from the fact that original client (normally) runs at all times and appears
to keep FAHlog.txt file descriptor open. What it means is, yeah, sure, you can append
data to the file but as offset tied to original client's fd remains unchanged, subsequent
updates (made by original client) will overwrite any appended data.

I don't have evidence this *will* happen -- call it a word of caution.


Re second item
It has been suggested before -- now I recall another bit that requires verification (I'll check
it with next fresh WU). I *think* client reads contents of client.cfg in at startup and
keeps them in memory, altering as necessary . What I think it also does is: it writes out
those contents to client.cfg whenever you stop the client (thus, again, discarding any
manually made changes*).

*) unless of course, it gets killed by SIGKILL (-9) ... ugh


tear
One man's ceiling is another man's floor.
Image
tear
Posts: 254
Joined: Sun Dec 02, 2007 4:08 am
Hardware configuration: None
Location: Rocky Mountains

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by tear »

tear wrote:(...) (I'll check
it with next fresh WU). I *think* client reads contents of client.cfg in at startup and
keeps them in memory, altering as necessary . What I think it also does is: it writes out
those contents to client.cfg whenever you stop the client (thus, again, discarding any
manually made changes*).
O.K. it doesn't do so. There's still one scenario to check (when langouste does not
kick in): autosend invoked while folding. I'll let you know.


tear
One man's ceiling is another man's floor.
Image
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by bruce »

Appending data to client.cfg is potentially dangerous. It's a machine-readable file and the routine in FAH that parses it works fine if the file is unaltered, but if you do make changes that stretch it's format in some unexpected way, the next time the client reads it it's likely to decide that the file is corrupt and reinitialize it to a known state. If that happens, it will be a state that you DO NOT WANT.
tear
Posts: 254
Joined: Sun Dec 02, 2007 4:08 am
Hardware configuration: None
Location: Rocky Mountains

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by tear »

There are many "conclusions" that I attribute to incomplete understanding of how client uses client.cfg.

"don't touch the file or everything will collapse" is one of them.


Cheers!
tear
One man's ceiling is another man's floor.
Image
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by bruce »

"Don't touch the file or everything will collapse" is actually true if you're a Windows user and you use a Windows text editor which messes with the end-of-line characters. You'll note that I did not say that in this forum. I simply said that you need to be careful and explained why.
tear
Posts: 254
Joined: Sun Dec 02, 2007 4:08 am
Hardware configuration: None
Location: Rocky Mountains

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by tear »

I'll be very careful!
One man's ceiling is another man's floor.
Image
p2501
Posts: 42
Joined: Sun May 04, 2008 9:10 am

Re: Langouste -- WU upload/download de-coupler [Linux only]

Post by p2501 »

Hello, I'll have to say thanks again for your effort with langouste, I also installed it on my new i7 860 and now it works gracefully there too.

I have to add a small bug report though (might also be a problem in the older release).
Because of the ext4 problems I partitioned sda with an ext2 partition for folding in mind.
I used /fold directly as fah client directory (i.e. /fold/work, /fold/langoustehelper.sh, etc.) and put the langouste temp dir (/fold/tmp) and the langouste folder (/fold/langouste3-0.12.2) on that partition/folder, so that the forked clients would also benefit from it being etx2.
That setting though caused langouste to fall into some kind of recursion error, it noticed that the client wanted to upload results, blocked that as expected and then produced several /tmp and working directories within /fold/tmp: /fold/tmp/pid/tmp/pid/tmp/pid...
Stupid me deleted all that, so you'll have to believe me and my jumbled bug report, I'll answer what I can though should you have questions.
Since I smelled that my strange setup would very likely be the cause I now have
/fold/fb (fah client)
/fold/langouste3-0.12.2
/fold/tmp (langouste temp), which works perfectly!! :)

Your work really is a great help for the community! *thumbs up* :biggrin:
Post Reply