Langouste -- WU upload/download de-coupler (+upload capping)

This forum contains information about 3rd party applications which may be of use to those who run the FAH client and one place where you might be able to get help when using one of those apps.

Moderator: Site Moderators

Nathan_P
Posts: 1180
Joined: Wed Apr 01, 2009 9:22 pm
Hardware configuration: Asus Z8NA D6C, 2 x5670@3.2 Ghz, , 12gb Ram, GTX 980ti, AX650 PSU, win 10 (daily use)

Asus Z87 WS, Xeon E3-1230L v3, 8gb ram, KFA GTX 1080, EVGA 750ti , AX760 PSU, Mint 18.2 OS

Not currently folding
Asus Z9PE- D8 WS, 2 E5-2665@2.3 Ghz, 16Gb 1.35v Ram, Ubuntu (Fold only)
Asus Z9PA, 2 Ivy 12 core, 16gb Ram, H folding appliance (fold only)
Location: Jersey, Channel islands

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by Nathan_P »

IF the WU are still in the work directory, shut down the client and restart with the -configonly flag, tell it not to use a proxy and then restart the client with the -sendall switch. It should send the outstanding WU.

I would then completely remove langouste, and recomplie/reinstall.
Image
tear
Posts: 254
Joined: Sun Dec 02, 2007 4:08 am
Hardware configuration: None
Location: Rocky Mountains

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by tear »

Thanks for the report, wombat.

Do you still have complete client directory? (w/unsent results, that is)
If so, can you please keep it (make a copy or whatnot) and make it somehow
available to me? (could set FTP up for you or something)

I'd like to understand what happened here but if we wipe everything there
will be nothing to work with...

I can't see a fault in the log excerpt... it does look like remote end closes your connection
for some reason.

Is it reproducible? If so, can you redo the test but this time add "-L langouste.log"
command-line parameter so we capture our log from the very beginning and also
save it to a file.

Also, FYI, w/-send all Langouste doesn't decouple but acts like regular HTTP proxy
so you will not see any new files in /dev/shm/.
One man's ceiling is another man's floor.
Image
tofuwombat
Posts: 19
Joined: Mon Nov 22, 2010 4:06 pm

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by tofuwombat »

Yes, VERY repeatable. :(
that log starts like this

Code: Select all

[1333824844.882126] Langouste3 0.15.7 (compiled Sat Apr  7 11:42:21 PDT 2012 by tofuwombat@schnellzug)
[1333824844.882241] Langouste3 comes with ABSOLUTELY NO WARRANTY; for details
[1333824844.882252] see `COPYING.txt' file located in source directory
[1333824844.882271] Default Langouste helper temp directory: /tmp/langouste-tofuwombat/
[1333824844.882503] Listening on 127.0.0.1:8880
[1333824844.882524] rl_update(): no RL connections
[1333824870.924517] Accepted connection from: 127.0.0.1:59223
[1333824870.928314] Found socket 125295
[1333824870.936626] PID 3897 FSUID: 1000, langouste FSUID: 1000
[1333824870.936645] PID for socket: 3897
[1333824870.936677] PID 3897: issending: 1
[1333824870.936693] rl_update(): no RL connections
[1333824870.936707] (0) local socket now ready
[1333824870.936741] (0) request: POST http://130.237.232.237:8080/ HTTP/1.1
Host: 130.237.232.237:8080
Connection: Keep-Alive
Content-Length: 222418772

[1333824870.936759] determine_host(): found host in request: '130.237.232.237:8080'
[1333824870.936769] determine_host(): found host in headers: '130.237.232.237:8080'
[1333824870.936780] (0) resolving '130.237.232.237:8080'
[1333824870.936827] (0) Connecting to: 130.237.232.237:8080
[1333824870.936870] (0) ratelimit: state 1
[1333824870.936882] rl_update(): no RL connections
[1333824870.936891] next connect() timeout in: 29.999979
[1333824880.144447] (0) remote socket now ready
[1333824880.144498] (0) Connected.
[1333824880.144509] nuploads++; now: 1
[1333824880.144541] (0) wrote 15928 bytes
[1333824880.144552] (0) remote send-q full
[1333824880.144567] rl_update(): no RL connections
[1333824880.537296] (0) remote socket now ready
[1333824880.537321] (0) wrote 456 bytes
[1333824880.537333] rl_update(): no RL connections
[1333824880.537373] (0) wrote 8232 bytes
[1333824880.537383] (0) remote send-q full
[1333824880.537393] rl_update(): no RL connections
[1333824880.929196] (0) remote socket now ready
[1333824880.929225] (0) wrote 8152 bytes
[1333824880.929236] rl_update(): no RL connections
[1333824880.929266] (0) wrote 7776 bytes
[1333824880.929276] (0) remote send-q full
[1333824880.929286] rl_update(): no RL connections
[1333824881.321835] (0) remote socket now ready
[1333824881.321866] (0) wrote 8608 bytes
[1333824881.321877] rl_update(): no RL connections
[1333824881.321902] (0) wrote 16384 bytes
[1333824881.321913] rl_update(): no RL connections
[1333824881.321945] (0) wrote 2520 bytes
[1333824881.321955] (0) remote send-q full
[1333824881.321966] rl_update(): no RL connections
[1333824881.976283] (0) remote socket now ready
[1333824881.976314] (0) wrote 13864 bytes
[1333824881.976326] rl_update(): no RL connections
[1333824881.976365] (0) wrote 13648 bytes
[1333824881.976375] (0) remote send-q full
[1333824881.976385] rl_update(): no RL connections
[1333824882.509852] (0) remote socket now ready
[1333824882.695075] (0) wrote 2736 bytes
[1333824882.704477] rl_update(): no RL connections
[1333824882.704539] (0) wrote 16384 bytes
[1333824882.704552] rl_update(): no RL connections
[1333824882.704573] (0) wrote 5496 bytes
[1333824882.704582] (0) remote send-q full
[1333824882.704593] rl_update(): no RL connections
[1333824883.103498] (0) remote socket now ready
[1333824883.103564] (0) wrote 10888 bytes
[1333824883.103580] rl_update(): no RL connections
[1333824883.103618] (0) wrote 16384 bytes
[1333824883.103629] rl_update(): no RL connections
[1333824883.103655] (0) wrote 8928 bytes
[1333824883.103665] (0) remote send-q full
[1333824883.103676] rl_update(): no RL connections
[1333824883.502582] (0) remote socket now ready
[1333824883.502628] (0) wrote 7456 bytes
[1333824883.502643] rl_update(): no RL connections
[1333824883.502682] (0) wrote 16384 bytes
[1333824883.502692] rl_update(): no RL connections
[1333824883.502722] (0) wrote 3672 bytes
[1333824883.502736] (0) remote send-q full
[1333824883.502752] rl_update(): no RL connections
[1333824883.920449] (0) remote socket now ready
[1333824883.920497] (0) wrote 12712 bytes
[1333824883.920512] rl_update(): no RL connections
[1333824883.920538] (0) wrote 16384 bytes
[1333824883.920570] rl_update(): no RL connections
[1333824883.920603] (0) wrote 1312 bytes
[1333824883.920613] (0) remote send-q full
[1333824883.920624] rl_update(): no RL connections
[1333824884.349489] (0) remote socket now ready
[1333824884.349518] (0) wrote 15072 bytes
[1333824884.349530] rl_update(): no RL connections
[1333824884.349553] (0) wrote 16384 bytes
[1333824884.349563] rl_update(): no RL connections
[1333824884.349588] (0) wrote 400 bytes
[1333824884.349597] (0) remote send-q full
[1333824884.349608] rl_update(): no RL connections
[1333824884.547429] (0) remote socket now ready
[1333824884.547454] (0) wrote 15984 bytes
[1333824884.547465] rl_update(): no RL connections
[1333824884.547497] (0) wrote 16384 bytes
[1333824884.547507] rl_update(): no RL connections
[1333824884.547527] (0) wrote 16384 bytes
[1333824884.547537] rl_update(): no RL connections
[1333824884.547567] (0) wrote 1928 bytes
[1333824884.547582] (0) remote send-q full
[1333824884.547598] rl_update(): no RL connections
[1333824885.141419] (0) remote socket now ready
[1333824885.141447] (0) wrote 14456 bytes
[1333824885.141458] rl_update(): no RL connections
[1333824885.141494] (0) wrote 16384 bytes
[1333824885.141505] rl_update(): no RL connections
[1333824885.141536] (0) wrote 16384 bytes


in the middle (80% through) transitioning from #5 to #6 it does this:

Code: Select all

 16384 bytes
[1333825385.434881] rl_update(): no RL connections
[1333825385.434927] (0) wrote 16384 bytes
[1333825385.434945] rl_update(): no RL connections
[1333825385.434988] (0) wrote 16384 bytes
[1333825385.435006] rl_update(): no RL connections
[1333825385.435045] (0) wrote 16384 bytes
[1333825385.435061] rl_update(): no RL connections
[1333825385.435107] (0) wrote 16384 bytes
[1333825385.435127] rl_update(): no RL connections
[1333825385.435157] (0) wrote 16384 bytes
[1333825385.435167] rl_update(): no RL connections
[1333825385.435196] (0) wrote 16384 bytes
[1333825385.435207] rl_update(): no RL connections
[1333825385.435234] (0) wrote 16384 bytes
[1333825385.435244] rl_update(): no RL connections
[1333825385.435271] (0) wrote 16384 bytes
[1333825385.435281] rl_update(): no RL connections
[1333825385.435303] (0) wrote 16384 bytes
[1333825385.435313] rl_update(): no RL connections
[1333825385.435342] (0) wrote 16384 bytes
[1333825385.435352] rl_update(): no RL connections
[1333825385.435379] (0) wrote 16384 bytes
[1333825385.435389] rl_update(): no RL connections
[1333825385.435410] (0) wrote 16384 bytes
[1333825385.435420] rl_update(): no RL connections
[1333825385.435450] (0) wrote 16384 bytes
[1333825385.435460] rl_update(): no RL connections
[1333825385.435488] (0) wrote 16384 bytes
[1333825385.435498] rl_update(): no RL connections
[1333825385.435520] (0) wrote 16384 bytes
[1333825385.435529] rl_update(): no RL connections
[1333825385.435558] (0) wrote 16384 bytes
[1333825385.435568] rl_update(): no RL connections
[1333825385.435602] (0) wrote 16384 bytes
[1333825385.435620] rl_update(): no RL connections
[1333825385.435657] (0) wrote 16384 bytes
[1333825385.435675] rl_update(): no RL connections
[1333825385.435721] (0) wrote 16384 bytes
[1333825385.435740] rl_update(): no RL connections
[1333825385.435781] (0) wrote 16384 bytes
[1333825385.435799] rl_update(): no RL connections
[1333825385.435832] (0) wrote 16384 bytes
[1333825385.435849] rl_update(): no RL connections
[1333825385.435886] (0) wrote 16384 bytes
[1333825385.435896] rl_update(): no RL connections
[1333825385.435926] (0) wrote 16384 bytes
[1333825385.435936] rl_update(): no RL connections
[1333825385.435961] (0) wrote 16384 bytes
[1333825385.435971] rl_update(): no RL connections
[1333825385.435997] (0) wrote 11712 bytes
[1333825385.436007] (0) remote send-q full
[1333825385.436017] rl_update(): no RL connections
[1333825448.631297] (0) Remote connection closed (rbsize: 0).
[1333825448.631709] (0) Local: received 15515648 bytes, sent 0 bytes
[1333825448.631731] (0) Remote: received 0 bytes, sent 15510976 bytes
[1333825448.631748] nuploads--; now: 0
[1333825448.631763] (0) Ratelimit: sent 15510976 byte(s) in 101.924 seconds, 152181 Bps (148.61 kBps)
[1333825448.631788] rl_update(): no RL connections
[1333825448.759196] Accepted connection from: 127.0.0.1:43994
[1333825448.763303] Found socket 125817
[1333825448.773254] PID 3897 FSUID: 1000, langouste FSUID: 1000
[1333825448.773281] PID for socket: 3897
[1333825448.773315] PID 3897: issending: 1
[1333825448.773339] rl_update(): no RL connections
[1333825448.773358] (0) local socket now ready
[1333825448.773391] (0) request: POST http://130.237.165.141:80/ HTTP/1.1
Host: 130.237.165.141:80
Connection: Keep-Alive
Content-Length: 100070147

[1333825448.773409] determine_host(): found host in request: '130.237.165.141:80'
[1333825448.773420] determine_host(): found host in headers: '130.237.165.141:80'
[1333825448.773430] (0) resolving '130.237.165.141:80'
[1333825448.773447] (0) Connecting to: 130.237.165.141:80
[1333825448.773562] (0) ratelimit: state 1
[1333825448.773575] rl_update(): no RL connections
[1333825448.773585] next connect() timeout in: 29.999978
[1333825448.966883] (0) remote socket now ready
[1333825448.966946] (0) Connected.
[1333825448.966957] nuploads++; now: 1
[1333825448.967001] (0) wrote 15928 bytes
[1333825448.967012] (0) remote send-q full
[1333825448.967032] rl_update(): no RL connections
[1333825449.159950] (0) Remote connection closed (rbsize: 0).
[1333825449.160027] (0) Local: received 16384 bytes, sent 0 bytes
[1333825449.160043] (0) Remote: received 0 bytes, sent 15928 bytes
[1333825449.160053] nuploads--; now: 0
[1333825449.160062] (0) Ratelimit: sent 15928 byte(s) in 0.193 seconds, 82528 Bps (80.59 kBps)
[1333825449.160074] rl_update(): no RL connections
[1333825449.160210] Accepted connection from: 127.0.0.1:43996
[1333825449.164051] Found socket 125820
[1333825449.173869] PID 3897 FSUID: 1000, langouste FSUID: 1000
[1333825449.173896] PID for socket: 3897
[1333825449.173927] PID 3897: issending: 1
[1333825449.173951] rl_update(): no RL connections
[1333825449.173969] (0) local socket now ready
[1333825449.174005] (0) request: POST http://130.237.165.141:8080/ HTTP/1.1
Host: 130.237.165.141:8080
Connection: Keep-Alive
Content-Length: 100070147

[1333825449.174021] determine_host(): found host in request: '130.237.165.141:8080'
[1333825449.174031] determine_host(): found host in headers: '130.237.165.141:8080'
[1333825449.174041] (0) resolving '130.237.165.141:8080'
[1333825449.174058] (0) Connecting to: 130.237.165.141:8080
[1333825449.174141] (0) ratelimit: state 1
[1333825449.174151] rl_update(): no RL connections
[1333825449.174161] next connect() timeout in: 29.999980
[1333825449.366263] (0) remote socket now ready
[1333825449.366308] (0) Connected.
[1333825449.366319] nuploads++; now: 1
[1333825449.366354] (0) wrote 15928 bytes
[1333825449.366365] (0) remote send-q full
[1333825449.366384] rl_update(): no RL connections
[1333825449.563347] (0) Remote connection closed (rbsize: 0).
[1333825449.563410] (0) Local: received 16384 bytes, sent 0 bytes
[1333825449.563425] (0) Remote: received 0 bytes, sent 15928 bytes
[1333825449.563435] nuploads--; now: 0
[1333825449.563444] (0) Ratelimit: sent 15928 byte(s) in 0.197 seconds, 80852 Bps (78.95 kBps)
[1333825449.563460] rl_update(): no RL connections
[1333825449.586198] Accepted connection from: 127.0.0.1:43998
[1333825449.590431] Found socket 125825
[1333825449.600458] PID 3897 FSUID: 1000, langouste FSUID: 1000
[1333825449.600484] PID for socket: 3897
[1333825449.600516] PID 3897: issending: 1
[1333825449.600539] rl_update(): no RL connections
[1333825449.600558] (0) local socket now ready
[1333825449.600591] (0) request: POST http://128.143.231.202:8080/ HTTP/1.1
Host: 128.143.231.202:8080
Connection: Keep-Alive
Content-Length: 11287473

[1333825449.600607] determine_host(): found host in request: '128.143.231.202:8080'
[1333825449.600618] determine_host(): found host in headers: '128.143.231.202:8080'
[1333825449.600628] (0) resolving '128.143.231.202:8080'
[1333825449.600645] (0) Connecting to: 128.143.231.202:8080
[1333825449.600734] (0) ratelimit: state 1
[1333825449.600746] rl_update(): no RL connections
[1333825449.600756] next connect() timeout in: 29.999979
[1333825449.693718] (0) remote socket now ready
[1333825449.693749] (0) Connected.
[1333825449.693762] nuploads++; now: 1
[1333825449.693814] (0) wrote 15928 bytes
[1333825449.693823] (0) remote send-q full
[1333825449.693835] rl_update(): no RL connections
[1333825449.872938] (0) remote socket now ready
[1333825449.872969] (0) wrote 456 bytes
[1333825449.872980] rl_update(): no RL connections
[1333825449.873046] (0) wrote 16384 bytes
[1333825449.873057] rl_update(): no RL connections
[1333825449.873080] (0) wrote 15472 bytes
[1333825449.873089] (0) remote send-q full
[1333825449.873099] rl_update(): no RL connections
[1333825449.964282] (0) remote socket now ready
[1333825449.964303] (0) wrote 912 bytes
[1333825449.964316] rl_update(): no RL connections
[1333825449.964383] (0) wrote 16384 bytes
[1333825449.964394] rl_update(): no RL connections
[1333825449.964444] (0) wrote 16384 bytes
[1333825449.964454] rl_update(): no RL connections
[1333825449.964479] (0) wrote 16384 bytes
[1333825449.964490] rl_update(): no RL connections
[1333825449.964513] (0) wrote 2064 bytes
[1333825449.964523] (0) remote send-q full
[1333825449.964533] rl_update(): no RL connections
[1333825450.061420] (0) remote socket now ready
[1333825450.061449] (0) wrote 14320 bytes
[1333825450.061461] rl_update(): no RL connections
[1333825450.061502] (0) wrote 16384 bytes
[1333825450.061512] rl_update(): no RL connections
[1333825450.061542] (0) wrote 6944 bytes
[1333825450.061552] (0) remote send-q full
[1333825450.061562] rl_update(): no RL connections
[1333825450.151839] (0) remote socket now ready
[1333825450.151865] (0) wrote 9440 bytes
[1333825450.151875] rl_update(): no RL connections
[1333825450.151907] (0) wrote 16384 bytes
[1333825450.151917] rl_update(): no RL connections
[1333825450.151944] (0) wrote 16384 bytes
and ends like this:

Code: Select all

[1333825477.618157] rl_update(): no RL connections
[1333825477.618178] (0) wrote 16384 bytes
[1333825477.618187] rl_update(): no RL connections
[1333825477.618209] (0) wrote 16384 bytes
[1333825477.618218] rl_update(): no RL connections
[1333825477.618240] (0) wrote 16384 bytes
[1333825477.618249] rl_update(): no RL connections
[1333825477.618269] (0) wrote 16384 bytes
[1333825477.618279] rl_update(): no RL connections
[1333825477.618299] (0) wrote 16384 bytes
[1333825477.618309] rl_update(): no RL connections
[1333825477.618330] (0) wrote 15405 bytes
[1333825477.618340] rl_update(): no RL connections
[1333825479.988961] rl_update(): no RL connections
[1333825479.989244] rl_update(): no RL connections
[1333825479.989259] (0) Remote connection closed (rbsize: 0).
[1333825479.989298] (0) Local: received 11287597 bytes, sent 615 bytes
[1333825479.989317] (0) Remote: received 615 bytes, sent 11287597 bytes
[1333825479.989328] nuploads--; now: 0
[1333825479.989337] (0) Ratelimit: sent 11287597 byte(s) in 30.295 seconds, 372589 Bps (363.85 kBps)
[1333825479.989348] rl_update(): no RL connections
Happy to send the file(s), let me know where. #5 is 212MB; #6 is 95MB; from one attempt that log is 514KB and the forum won't let me paste all of it here

I DO have copies of the stranded result files, but there have been several half-baked attempts to prove that langouste is configurable by me, where I cycled through: make clean; make dep; make while sprinkling langouste-helper.sh files about.


With the /dev/shm config, it feels like I cannot change the TEMPDIR. When I run it it reports that it is using the OLD one in /tmp. . . . like it won't listen to me. I don't see ANY langouste files in /dev/shm
tofuwombat
Posts: 19
Joined: Mon Nov 22, 2010 4:06 pm

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by tofuwombat »

Nathan_P wrote:IF the WU are still in the work directory, shut down the client and restart with the -configonly flag, tell it not to use a proxy and then restart the client with the -sendall switch. It should send the outstanding WU.

I would then completely remove langouste, and recomplie/reinstall.
Yes, I did try this. if that had worked, I would have just stepped away from langouste.

But now I'm curious . . .
tofuwombat
Posts: 19
Joined: Mon Nov 22, 2010 4:06 pm

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by tofuwombat »

It looks like I MIGHT need to research BIOS && OS settings: :oops:

Code: Select all

tofuwombat@schnellzug:~$ uname -a
Linux schnellzug 2.6.35-22-generic #33-Ubuntu SMP Sun Sep 19 20:32:27 UTC
2010 x86_64 GNU/Linux
tofuwombat@schnellzug:~$ dmesg | grep RAM
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] AMI BIOS detected: BIOS may corrupt low RAM, working around
it.
[    0.000000] WARNING: BIOS bug: CPU MTRRs don't cover all of memory,
losing 61936MB of RAM.
[    0.000000] modified physical RAM map:
[    0.000000] modified physical RAM map:
[    0.000000] RAMDISK: 37570000 - 37ff0000
[    0.000000]   #2 [0037570000 - 0037ff0000]         RAMDISK
[    0.000000]   #9 [0000010000 - 0000012000]      TRAMPOLINE
[   10.983878] reserve RAM buffer: 000000000009d400 - 000000000009ffff
[   10.983878] reserve RAM buffer: 00000000dfea0000 - 00000000dfffffff
tofuwombat@schnellzug:~$


tofuwombat@schnellzug:~$ dmesg | grep WARNING
[    0.000000] WARNING: BIOS bug: CPU MTRRs don't cover all of memory,
losing 61936MB of RAM.
[    0.000000] WARNING: at
/build/buildd/linux-2.6.35/arch/x86/kernel/cpu/mtrr/cleanup.c:971
mtrr_trim_uncached_memory+0x2d8/0x303()
tofuwombat@schnellzug:~$
My favorite line is: "losing 61936MB of RAM"

I'm guessing differing assumptions about memory maps could cause LOTS of problems . . .

Any Idea if these errors (and IOMMU) could cause the problems with my stranded work units?

Any one have favorite BIOS/OS settings for Tyan S8812(v105 BIOS) for kubuntu 10.10/kraken/langouste/fah6.34 -smp 64? :)

I'll start looking at a manual . . .
tear
Posts: 254
Joined: Sun Dec 02, 2007 4:08 am
Hardware configuration: None
Location: Rocky Mountains

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by tear »

At [H], we've got G34 performance checklist: http://hardforum.com/showthread.php?t=1669698

To answer your question, no, these settings should not affect Langouste's operation in principle.

Working on FTP setup, will PM the details.


Thanks,
tear
One man's ceiling is another man's floor.
Image
Klue22
Posts: 5
Joined: Mon May 21, 2012 8:08 am

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by Klue22 »

What would be required to get the langouste to accept connections from other computers?
tear
Posts: 254
Joined: Sun Dec 02, 2007 4:08 am
Hardware configuration: None
Location: Rocky Mountains

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by tear »

It depends on what you are trying to accomplish...

Currently, decoupling requires an instance of Langouste per machine.

Upload capping could be made to work with one instance per... subnet (or whatnot)
but you'd lose decoupling feature.

What is your use case?
One man's ceiling is another man's floor.
Image
Klue22
Posts: 5
Joined: Mon May 21, 2012 8:08 am

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by Klue22 »

Basically my internet connection is spotty enough that I sometimes can't upload WUs before the two hour timeout. My thinking was that by VPNing my dedicated sever (at a data center) to my home network that I could run langouste on the server to ensure that WU uploads wouldn't time out.

So (in my head) the process goes like this:
1. WU finishes, folding computer tries to connect through langouste
2. Langouste pulls the WU files from home computer to the server
3. Langouste uploads the WU fast enough that stanford servers don't time out

I don't know the inner workings of langouste but I assume the server would have to be able to access the folding directory on my home computer (hence the VPN).
tear
Posts: 254
Joined: Sun Dec 02, 2007 4:08 am
Hardware configuration: None
Location: Rocky Mountains

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by tear »

Right, though you'd still need a method to reliably relay the data between
two locations. As both locations are yours 2-hour limit would not apply.

Is your link just borderline slow or unreliable (drops) ?

Running remote instance of Langouste could be a bit challenging. In your
environment.... Would you be able to, say, ssh/scp to your remote machine?
One man's ceiling is another man's floor.
Image
Klue22
Posts: 5
Joined: Mon May 21, 2012 8:08 am

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by Klue22 »

Its mostly just slow (since i live out in the sticks).
I actually think I got it mostly. I changed this line (in langouste.c):

listen_address.s_addr = htonl(INADDR_LOOPBACK);
to
listen_address.s_addr = htonl(INADDR_ANY);

before compiling it. It seems to work as long as I have the two computers are VPNed together (using hamachi) and the folding directory is set to be shared. I haven't tried it with the large WUs yet (just a few small ones for testing). How would I go about getting a windows version? Currently I'm having to VM linux for just langouste.
tear
Posts: 254
Joined: Sun Dec 02, 2007 4:08 am
Hardware configuration: None
Location: Rocky Mountains

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by tear »

I think there's a bit more that would need to be done.

Problem with listening on 0.0.0.0 is that when remote client connects to Langouste,
Langouste is not able to determine (remote) cllient's PID and, consequently, working
directory of the client -- decoupling is turned off and Langouste acts like regular HTTP
proxy and offers no benefits.

If you look at Langouste log (with INADDR_ANY mod) you'll see that each remote
connection is associated with a warning, e.g.:

Code: Select all

Thu Jun  7 15:21:43 2012 Accepted connection from: 10.100.166.196:59918
Thu Jun  7 15:21:43 2012 WARNING: couldn't find PID for socket
What I think needs to happen in your case is:
1. WU results (along with client copy) need to be copied completely over to remote location
2. "Background" client instance needs to be run on remote location

(1) could be facilitated by mounting remote fs and pointing langouste's temp dir
  to it in the helper script
(2) could be facilitated by modifying the helper script further and using ssh with pubkey
  authentication to launch remote instance of Langouste

One would need to additionally make sure that only one instance of Langouste runs on
remote machine at a time.
One man's ceiling is another man's floor.
Image
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by bruce »

You could also switch to V7. Both uploading and downloading are decoupled from processing.

Nothing will really fix a bad connection, but once the next WU gets started, the uploads can be retried using either [V6+Langouste] or [V7]. Ultimately, I'm not sure I understand why transferring a WU from point A to point B and then to point C is going to correct for a cruddy connection. The VPN has to go across two connections plus a third one to get it where it's going.
tear
Posts: 254
Joined: Sun Dec 02, 2007 4:08 am
Hardware configuration: None
Location: Rocky Mountains

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by tear »

V7 won't change 2h upload deadline which is what Klue is aiming to mitigate.

The idea is that results are relayed to the middle man in a state-less manner. Think of it as automatic sneakernetting.
Last edited by tear on Thu Jun 07, 2012 10:18 pm, edited 1 time in total.
One man's ceiling is another man's floor.
Image
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Langouste -- WU upload/download de-coupler (+upload capp

Post by bruce »

tear wrote:V7 won't change 2h upload deadline which is what Klue is aiming to mitigate.
True, but I don't see how that's any different with V6 -- unless he puts a cap on the bandwidth, which will only make it worse.

The question is what to do about a crappy connection.
Post Reply