UPDATE: Langouste 0.15.7 is now available. It resolves two major (though extremely sporadic) Linux reliability issues.
Upgrade is strongly recommended to all Langouste Linux users.
Linux users: when configuring the client always use 127.0.0.1 as the proxy address;
do _not_ use localhost as it may render client unable to contact Langouste (and, subsequently, upload or download any WUs)
Langouste does upload capping (see section 6)!
Langouste works on Microsoft Windows systems: Windows users are encouraged to read sections 1, 2, 3 and 5
Pull from http://darkswarm.org/langouste3-0.15.7.zip or http://darkswarm.org/langouste3-0.15.7.tar.gz.
- Code: Select all
$ cat README.txt
Contents:
0. License
1. What does Langouste do?
2. How is Langouste useful?
3. Supported configurations (PLEASE READ!)
4. Linux information
4.1. Building Langouste
4.2. Using Langouste
4.3. Upgrading Langouste
4.4. How do I know it's working?
4.5. For advanced users
5. Windows information
5.1. Using Langouste
5.2. Upgrading Langouste
5.3. How do I know it's working?
6. Rate limiting (upload capping) feature
6.1. Rate limiting with decoupling (1 Langouste per machine)
7. Known issues and caveats
8. Support
9. Credits
0. License
Langouste is licensed under GNU General Public License version 2.
See COPYING.txt for details.
1. What does Langouste do?
Upon client's attempt to contact collection server (with a WU to return)
it forks a copy of the client (in temporary directory) with -send
parameter and terminates connection "original" client was attempting to
make. If forked client is able to send data back, corresponding
wuresults_*.dat file is wiped from original client's work directory
(so original client's autosend bails out and removes it from queue).
2. How is Langouste useful?
Its intended purpose is to make WU download and return processes
asynchronous (in other words to enable simultaneous downloads and uploads).
Processed SMP and SMP -bigadv WUs are usually large (25-100MB). When WU
processing is complete client sends back the results but new WU is not
retrieved until after upload is complete.
On relatively fast links (1Mbit up) bigadv upload takes ~12 minutes (+5
extra minutes of FAH client/server processing).
Moreover, due to client deficiencies (see
http://foldingforum.org/viewtopic.php?f=44&t=10615) results file cleaning
takes additional time (depending on underlying filesystem -- between
several and 20+ minutes).
As Langouste decouples upload and download processes a new WU can be
downloaded while WU upload continues in background thus saving time.
Time that can be spent on simulation.
3. Supported configurations (PLEASE READ!)
Langouste is supported on modern Linux distributions running Folding@Home
clients 6.02 (32-bit) and 6.34 (64-bit).
Langouste is also supported on 32- and 64-bit Windows running
Folding@Home clients (console, console+GPU, console+SMP, systray, systray+GPU)
6.23, 6.30, 6.41, _not_ in service mode.
Minimal requirement is Windows XP with Service Pack 2. Windows Vista and 7
are also supported.
4. Linux information
4.1. Bulding Langouste
0. Make sure gcc, make and appropriate development packages are installed
1. Run "make dep"
2. Run "make"
All that should create "langouste3" binary
4.2. Using Langouste
1. Copy helper script (from dist/linux/ directory) to FAH client's
directory
2. Pick a non-used port for Langouste to use (I use 8880)
3. Start Langouste*,**: ./langouste3 -l port-from-step-2
4. Reconfigure FAH client (-config or -configonly)
- set proxy host to 127.0.0.1 (do not leave "localhost" there)
- set proxy port to port from step 2**
5. Enjoy
*) if you wish to run Langouste in the backgound (otherwise you need
to keep Langouste terminal window open at all times) add -D option
to Langouste's command line, e.g. ./langouste3 -l port-from-step-2 -D
**) it is _critical_ Langouste is run as the same user as FAH client(s)
(it has no effect otherwise)
***) Langouste is designed to handle multiple clients (on same machine) --
running single instance is good enough
4.3. Upgrading Langouste
0. Pick a time when FAH client is not sending or receiving data (shutting
client down is not necessary)
1. Terminate existing instance of Langouste
2. Start new instance of Langouste
3. Copy new helper script (from dist/linux/ directory) to FAH client
directory
4. Remove old Langouste directory/binary (for safety)
4.4. How do I know it's working?
First off, even if you intend to run Langouste as a daemon (in the
background) running it in the foreground for a while is recommended (just
to make sure things work as expected).
Anyway, the easiest way is to examine forked client's log at (or after)
the time WU is returned (/tmp/langouste directory). See reference log
in dist/linux/reflogs/ directory.
Langouste terminal output follows (informative).
Client attempts to return a WU:
Fri Mar 18 12:10:43 2011 Accepted connection from: 127.0.0.1:43144
Fri Mar 18 12:10:43 2011 PID for socket: 1640
Fri Mar 18 12:10:43 2011 PID 1640: issending: 0
Fri Mar 18 12:10:43 2011 ===> PID 1640 is (most likely) contacting WU server, content length: 101817447
Fri Mar 18 12:10:43 2011 ===> Helper pid: -1
Fri Mar 18 12:10:43 2011 ===> PID 1640: numcores: 0
Fri Mar 18 12:10:43 2011 ===> now: 1300471843, last helper launched at: 0
Fri Mar 18 12:10:43 2011 ===> Launching helper: '/proc/1640/cwd/langouste-helper.sh' (exe name: '/fah/clients/fah-6.34/fah6')...
Fri Mar 18 12:10:43 2011 ===> Forked 2433
Fri Mar 18 12:10:43 2011 (0) Local: received 16384 bytes, sent 0 bytes
Initial connection is terminated and a new client instance (set up to
return WU) is forked (via helper script -- PID 2433).
Helper output (normally mixed with Langouste output):
DIRNAME: /proc/1640/cwd
READLINK: /fah/clients/fah-6.34
BASENAME: langouste-helper.sh
/proc/1640/cwd/langouste-helper.sh: launching asynchronous part, using /fah/clients/fah-6.34/langouste-helper.sh
As original client doesn't know we're doing anything in the background it
continues its attempts to return WU (for a while) -- you should see the
following sequence several times:
Fri Mar 18 12:10:43 2011 Accepted connection from: 127.0.0.1:43145
Fri Mar 18 12:10:43 2011 PID for socket: 1640
Fri Mar 18 12:10:43 2011 PID 1640: issending: 0
Fri Mar 18 12:10:43 2011 ===> PID 1640 is (most likely) contacting WU server, content length: 101817447
Fri Mar 18 12:10:43 2011 ===> Helper pid: -1
Fri Mar 18 12:10:43 2011 ===> PID 1640: numcores: 0
Fri Mar 18 12:10:43 2011 ===> now: 1300471843, last helper launched at: 1300471843
Fri Mar 18 12:10:43 2011 ===> WARNING: can only launch one helper in 2 minutes (per client)
Fri Mar 18 12:10:43 2011 (0) Local: received 16384 bytes, sent 0 bytes
When it gives up WU return attempts it initiates download a new WU:
Fri Mar 18 12:10:44 2011 Accepted connection from: 127.0.0.1:43150
Fri Mar 18 12:10:44 2011 PID for socket: 1640
Fri Mar 18 12:10:44 2011 PID 1640: issending: 0
Fri Mar 18 12:10:44 2011 ===> PID 1640 is contacting main assignment server
Fri Mar 18 12:10:44 2011 (0) resolving 'assign.stanford.edu:8080'
Fri Mar 18 12:10:44 2011 (0) Connecting to: 171.67.108.200:8080
Fri Mar 18 12:10:44 2011 (0) Connected.
Fri Mar 18 12:10:44 2011 (0) Local connection closed (bsize: 0).
Fri Mar 18 12:10:44 2011 (0) Local: received 559 bytes, sent 396 bytes
Fri Mar 18 12:10:44 2011 (0) Remote: received 396 bytes, sent 559 bytes
Fri Mar 18 12:10:44 2011 Accepted connection from: 127.0.0.1:43152
Fri Mar 18 12:10:44 2011 PID for socket: 1640
Fri Mar 18 12:10:44 2011 PID 1640: issending: 0
Fri Mar 18 12:10:44 2011 ===> PID 1640 is (most likely) contacting WU server, content length: 512
Fri Mar 18 12:10:44 2011 (0) resolving '171.67.108.22:8080'
Fri Mar 18 12:10:44 2011 (0) Connecting to: 171.67.108.22:8080
Fri Mar 18 12:10:44 2011 (0) Connected.
One minute after first WU return attempt the forked (background) client
kicks in:
Fri Mar 18 12:11:44 2011 Accepted connection from: 127.0.0.1:36593
Fri Mar 18 12:11:44 2011 PID for socket: 2447
Fri Mar 18 12:11:44 2011 PID 2447: issending: 1
Fri Mar 18 12:11:44 2011 (1) resolving '171.67.108.22:8080'
Fri Mar 18 12:11:44 2011 (1) Connecting to: 171.67.108.22:8080
Fri Mar 18 12:11:44 2011 (1) Connected.
Original client completes download of new WU:
Fri Mar 18 12:13:46 2011 (0) Local connection closed (bsize: 0).
Fri Mar 18 12:13:46 2011 (0) Local: received 627 bytes, sent 25473261 bytes
Fri Mar 18 12:13:46 2011 (0) Remote: received 25473261 bytes, sent 627 bytes
And again, tries to return WU (one that's already being handled by forked
client) several times.
Fri Mar 18 12:13:46 2011 Accepted connection from: 127.0.0.1:36595
Fri Mar 18 12:13:46 2011 PID for socket: 1640
Fri Mar 18 12:13:46 2011 PID 1640: issending: 0
Fri Mar 18 12:13:46 2011 ===> PID 1640 is (most likely) contacting WU server, content length: 101817447
Fri Mar 18 12:13:46 2011 ===> Helper pid: 2447
Fri Mar 18 12:13:46 2011 ===> PID 1640: numcores: 0
Fri Mar 18 12:13:46 2011 ===> Helper (pid 2447) already running
Fri Mar 18 12:13:46 2011 ===> If you're sure that's not the case, please remove /tmp/fah/f1
Fri Mar 18 12:13:46 2011 (0) Local: received 16384 bytes, sent 0 bytes
Finally, forked (background) client completes the upload:
Fri Mar 18 12:18:55 2011 (1) Local connection closed (bsize: 0).
Fri Mar 18 12:18:55 2011 (1) Local: received 101817568 bytes, sent 636 bytes
Fri Mar 18 12:18:55 2011 (1) Remote: received 636 bytes, sent 101817568 bytes
Fri Mar 18 12:18:55 2011 (1) Ratelimit: sent 101817568 byte(s) in 431.722 seconds, 235840 Bps (230.31 kBps)
4.5. For advanced users
If your machine makes use of tmpfs (check 'df | grep /dev/shm') you can
ease the load on your hard-drive by editing TMPDIR variable in helper
scripts and setting it to /dev/shm/langouste-$USER (instead of default
/tmp/langouste-$USER).
Be advised, TMPDIR must _not_ lie within client's directory; hell will
break loose otherwise:
http://foldingforum.org/viewtopic.php?f=14&t=11615&start=30#p126083
http://foldingforum.org/viewtopic.php?f=14&t=11615&start=45#p126147
5. Windows information
5.1. Using Langouste
1. Unzip Langouste archive
2. Copy helper batch file (from dist\win32\ directory) to FAH client's
directory (console) or, if using systray client, to Folding@Home data
files directory
3. Pick a non-used port for Langouste to use (I use 8880)
4. Start cmd.exe
5. Change directory to dist\win32\ subdirectory of the unzipped archive
6. Start Langouste*: langouste3-0.15.7.exe -l port-from-step-3
7. Reconfigure FAH client:
- set proxy host to 127.0.0.1 (do not leave "localhost" there)
- set proxy port to port from step 3**
8. Enjoy
*) it is _critical_ Langouste is run as the same user as FAH client(s)
(it has no effect otherwise)
**) Langouste is designed to handle multiple clients (on same machine) --
running single instance is good enough
5.2. Upgrading Langouste
0. Pick a time when FAH client is not sending or receiving data (shutting
client down is not necessary)
1. Terminate existing instance of Langouste with Control+C
2. Start new instance of Langouste
3. Copy new helper script (from dist\win32\ directory) to FAH client
directory (console) or, if using systray client, to Folding@Home data
files directory
4. Remove old Langouste directory/binary (for safety)
5.3. How do I know it's working?
This section is yet to be written. For now, see reference FAH client,
Langouste and Langouste helper logs (captured with version 0.14.1)
in dist\win32\reflogs\ (courtesy of DonMarkoni).
6. Rate limiting (upload capping) feature
Langouste releases 0.15 and later come with a feature that enables user
to limit upload rates.
Capping uploads to as much as 90% of upstream bandwith makes heaven and
earth difference with interactive sessions (ssh et al.) and regular "web
browsing" too -- DNS queries and TCP ACKs require a bit of (ideally
low-latency) upstream bandwidth.
6.1. Rate limiting with decoupling (1 Langouste per machine)
How to? Just add -r <rate-in-bytes-per-second> parameter to you current
command line; see following example (and terminal log):
$ ./langouste3 -l 8880 -r 28160
Tue Aug 3 11:23:11 2010 Langouste3 0.15.5 (compiled Tue Aug 3 11:22:59 MDT 2010 by fah@tentacle)
Tue Aug 3 11:23:11 2010 Langouste3 comes with ABSOLUTELY NO WARRANTY; for details
Tue Aug 3 11:23:11 2010 see `COPYING' file located in source directory
Tue Aug 3 11:23:11 2010 Default Langouste helper temp directory: /tmp/langouste-fah/
Tue Aug 3 11:23:11 2010 Ratelimit: Output rate: 28160 bytes/s (27.50 kBps)
Tue Aug 3 11:23:11 2010 Listening on 127.0.0.1:8880
(...)
Tue Aug 3 14:16:03 2010 (0) Ratelimit: sent 100029927 byte(s) in 3561.781 seconds, 28084 Bps (27.42 kBps)
IMPORTANT CAVEAT: multiple machines in the same network won't know about
others' existence;
two (or more) machines returning WUs simultaneously may
(and normally will) exceed limits imposed on any single
machine;
on the other hand, turnaround time of SMP and SMP bigadv
WUs is fairly long so probability of such "clash" isn't
really very high
7. Known issues and caveats
I-1. Using Langouste results in local WU count not being updated; note
that Stanford statistics are _not_ affected by this issue.
Report: http://foldingforum.org/viewtopic.php?f=14&t=11615&start=30#p125534
Comments: updating local WU count is tricky and possible
solutions are prone to variety of race-conditions;
I'm currently reluctant to resolving the issue
at the cost of reliability
8. Support
Using foldingforum.org is recommended. Please post in this thread:
http://foldingforum.org/viewtopic.php?f=14&t=11615
9. Credits
Langouste was written and is maintained by Kris Rusocki <kszysiu@gmail.com>
Special thanks go to:
Punchy -- for a great deal of troubleshooting assistance, resolving
Langouste's issues on SLES 10 and excellent Windows port
alpha-test feedback
pholcman -- for excellent feedback on rate limiting feature and
general troubleshooting assistance (esp. case with
Langouste residing on other-than-system volume)
DonMarkoni -- for excellent Windows port alpha-test feedback
metal03326 -- for excellent Windows port alpha-test feedback
Blasphemous Cannibal -- for extraordinary amount of persistence
while troubleshooting "infinite loop" issue
Hyperlife -- for help with IPv6 localhost name resolution issue
$
Debian/Ubuntu step-by-step build instructions can be found here (with credit for bollix47).
Change log (since first official release):
2011-04-25 -- New release (langouste3-0.15.7)
-- resolves two major Linux issues; upgrade strongly recommended
-- Linux: resolves issue that could cause Langouste to completely
stop responding once helper script has started (WU return);
such situation would require restart of Langouste
-- Linux: resolves issue that could prevent Langouste from correctly
launching background client; such situation would result
in delayed WU return
-- includes minor documentation update
2010-08-30 -- New release (langouste3-0.15.6)
-- includes two reliability bugfixes (Windows); upgrade is
recommeneded
-- Windows: resolves an issue that could result in Langouste
ceasing to operate and going into infinite loop under
certain conditions (thanks to Blasphemous Cannibal for
troubleshooting assistance)
-- Windows: plugs process handle leak
-- minor cosmetic improvements (compiler warnings, messages)
-- minor documentation tweaks
2010-08-03 -- New release (langouste3-0.15.5)
-- adds reliability improvements: upgrade is recommended
-- Langouste now implements own TCP connection timeout (30s) instead
of relying on FAH client timeout (1h)
-- Linux: improves helper behaviour in a scenario when original client
gets shut down when helper is running
-- Linux: improves behaviour with paths containing whitespace characters
-- Windows: adds support for Langouste residing on drive other system
drive
-- non-zero verbosity levels now use epoch timestamps with microsecond
resolution
-- minor code optimizations and cosmetic fixes
-- adds .txt extension to README, CHANGES and COPYING files (for Windows
users)
-- ACLs now accept CIDR masks
-- ACLs and network mode disabled until release-ready
-- NEW: introduces upload capping feature (see section 6 of README.txt
for more information)
2010-07-19 -- New release (langouste3-0.14.4)
-- Maintenance release
-- Linux: improves robustness in corner-case path and/or client binary names;
updated helper script _must_ be used
-- Windows: fixes (cosmetic) issue that could result in Langouste producing
Winsock errors at shutdown
-- adds missing parameter to Usage/Help screen
-- misc documentation updates
2010-07-18 -- New release (langouste3-0.14.3)
-- Linux: maintenance release; upgrade if you're experiencing issues
related to item below
-- resolves issue that could result in Langouste not being able download
data
-- cleanup: closing all fds after fork has been replaced with FD_CLOEXEC
-- b&w: RX/TX byte counters have been added
-- some messages are now much more informative
-- minor documentation updates have been committed
-- Langouste has been reworked to accommodate Windows porting (porting.c,
core*.c)
-- Langouste is now available for Microsoft Windows platforms!; see README
for details
2010-01-23 -- New release (langouste3-0.12.4)
-- maintenance release; upgrade if you're experiencing issues related
to items below
-- adds logic preventing langouste from handling clients run by other
users
-- adds ownership check of /tmp/fah directory
-- changes langouste TMPDIR to /tmp/langouste-$USER
-- fixes a bug that caused HTTP requests without Content-Length header
be interpreted as a requests with non-zero Content-Length (bug affected
standalone proxy mode; no interference with FAH)
-- cosmetic code improvements
-- minor Makefile tweaks
-- minor documentation updates
2010-01-01 -- New release (langouste3-0.12.2)
-- maintenance release; upgrade if you're experiencing issues related
to items below
-- adds support for SLES 10 (SP2); it uses a different format of socket
entries in /proc/pid/fd (thanks go to bingo-dog for troubleshooting and
initial fix)
-- resolves a bug in socket lookup code that could result (under
certain circumstances) in langouste not being able to find FAH
client's PID; this bug has not been seen in the wild
(thanks go to bingo-dog for identification and initial fix)
-- [advanced users] lazy signals finally work as expected
-- corrects a typo in an error message (found by bingo-dog)
-- tunes up the Makefile
-- adds clarifications to README (langouste needs to be run as the
same user as FAH client(s))
