Folding Forum

by **slegrand** » Tue Jul 01, 2008 10:06 pm

Subject says it all... I've just been given clearance to spill the beans...

But some caveats upfront..

1. Device 0 *must* be a supported CUDA and DirectX display device. It's a client quirk, and there is a workaround in development, but for now, those are the rules.

2. If all your GPUs are CUDA devices and they are all extended onto the desktop, you can just use -gpu n where n is the display device number and it will work just like an ATI client, which means follow the ATI multi-GPU FAQ and you'll be folding away.

3. In the event that 2 above is not the case, there is an override. Skip the -gpu flag and instead set the environment variable NV_FAH_DEVICE=n where n is the CUDA device you wish to use. You may find yourself also playing with CPU affinity here to get the right mix, but that's why we provided the power tools here so to speak. And also it provides a way for anyone with a tesla server/board out there to run F@H today.

3. set VAR=value is the proper syntax. If you put spaces in this, it won't work, i.e. set VAR = value is incorrect.

4. You need to set these variables either in a script that launches the client, or before you launch the client. Doing so after starting the client will have no effect until it is stopped and restarted.

I'm sure we're about to see some quirkiness, but that's what betas are all about... Have fun!

by **theheyes** » Tue Jul 01, 2008 10:10 pm

With two high performance GPUs sharing the same bus, are we likely to see diminishing returns adding more cards?

by **slegrand** » Tue Jul 01, 2008 10:32 pm

theheyes wrote:With two high performance GPUs sharing the same bus, are we likely to see diminishing returns adding more cards?

It ought to scale like crazy... The only thing that might stop that for now is our weird desktop perf issue, but once that's cleared, the whole thing runs as a closed system on the GPU, only periodically communicating data to the CPU every 12 to 30 seconds of real-time.

by **VijayPande** » Tue Jul 01, 2008 10:36 pm

We'd be curious to hear what people find. I suspect that one should have one core per GPU, so that means (for most people) no more than 8 GPUs = 4 x 9800GX2

by **slegrand** » Tue Jul 01, 2008 10:39 pm

VijayPande wrote:We'd be curious to hear what people find. I suspect that one should have one core per GPU, so that means (for most people) no more than 8 GPUs = 4 x 9800GX2

Once we clear up what's hogging core 0, that might not even be such a limiting factor - witness the 6% CPU load on some Vista clients... Something is going horribly wrong on XP and the good part about that is that once we fix that, I suspect CPU load will be moderate.

by **DonMarkoni** » Tue Jul 01, 2008 10:49 pm

YOU PEOPLE ARE CRAZY! NEW CORES, NEW CLIENTS EVERY FEW HOURS! I WANT TO SLEEP! :-)

Now, seriously, this is working great!
Now, seriously, this is working great!
If You see this twice, this is realy working great!

EDIT:
First G92 GPU using one CPU core folding project 5205 R1 C286 G0 ~6000ppd
Second G92 GPU using one CPU core folding project 5002 R2 C20 G149 ~6500ppd
Only two remaining CPU cores folding project 2665 R3 C824 G23 ~2000ppd

by **Bernie64** » Tue Jul 01, 2008 11:24 pm

So any GX2 is seen as one GPU to the client but is physically 2 GPU's?
:?:

EDIT: 14,500ppd on one mobo?

by **Leoslocks** » Tue Jul 01, 2008 11:44 pm

If it is the machine in his signature, it is 2x 8800GTS 512MB @ 800/2000/2100

Seriously, that was really funny Don.

1.07 *IS* *Multi-GPU*

No need to change the title in the 'other thread'.

by **Fadamor** » Tue Jul 01, 2008 11:48 pm

Leoslocks wrote:If it is the machine in his signature, it is 2x 8800GTS 512MB @ 800/2000/2100

Seriously, that was really funny Don.

1.07 *IS* *Multi-GPU*
No need to change the title in the 'other thread'.

Scott says to follow the ATI multi-gpu FAQ... I give. Where exactly is this FAQ?

by **leexgx** » Wed Jul 02, 2008 12:01 am

cool i will test right away

by **cbr1100xxVanish** » Wed Jul 02, 2008 12:03 am

It doesn't work for me.

Vista 64 / 1.07 / 177.35 / 2 x 8800M GTX.
One card works but the second gives unstable problems starting with a "mdrun_gpu returned -1"

Code: Select all: --- Opening Log file [July 2 04:19:57] # Windows Console GPU Edition ################################################# ############################################################################### Folding@Home Client Version 6.12beta8 http://folding.stanford.edu ############################################################################### ############################################################################### Launch directory: C:\Users\Harald\AppData\Roaming\Folding@home-gpu2 Arguments: -gpu 2 -local -verbosity 9 [04:19:57] - Ask before connecting: No [04:19:57] - User name: 8800MGTX_GPU (Team 74103) [04:19:57] - User ID: 2FDC27456A21EB8 [04:19:57] - Machine ID: 7 [04:19:57] [04:19:57] Loaded queue successfully. [04:19:57] Initialization complete [04:19:57] + Benchmarking ... [04:19:57] - Preparing to get new work unit... [04:19:57] - Autosending finished units... [04:19:57] + Attempting to get work packet [04:19:57] Trying to send all finished work units [04:19:57] + No unsent completed units remaining. [04:19:57] - Autosend completed [04:19:57] - Will indicate memory of 4093 MB [04:19:57] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 15, Stepping: 11 [04:19:57] - Connecting to assignment server [04:19:57] Connecting to http://assign-GPU.stanford.edu:8080/ [04:19:58] Posted data. [04:19:58] Initial: 40AB; - Successful: assigned to (171.64.65.20). [04:19:58] + News From Folding@Home: GPU folding beta [04:19:58] Loaded queue successfully. [04:19:58] Connecting to http://171.64.65.20:8080/ [04:19:59] Posted data. [04:19:59] Initial: 0000; - Receiving payload (expected size: 41482) [04:19:59] Conversation time very short, giving reduced weight in bandwidth avg [04:19:59] - Downloaded at ~81 kB/s [04:19:59] - Averaged speed for that direction ~50 kB/s [04:19:59] + Received work. [04:19:59] + Closed connections [04:19:59] [04:19:59] + Processing work unit [04:19:59] Core required: FahCore_11.exe [04:19:59] Core found. [04:19:59] Working on queue slot 06 [July 2 04:19:59] [04:19:59] + Working ... [04:19:59] - Calling 'FahCore_11.exe -dir work/ -suffix 06 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 1736 -version 612' [04:19:59] [04:19:59] *------------------------------* [04:19:59] Folding@Home GPU Core - Beta [04:19:59] Version 1.07 (Mon Jun 23 10:53:13 PDT 2008) [04:19:59] [04:19:59] Compiler : [04:19:59] Build host: amoeba [04:19:59] Preparing to commence simulation [04:19:59] - Looking at optimizations... [04:19:59] - Created dyn [04:19:59] - Files status OK [04:19:59] - Expanded 40970 -> 246249 (decompressed 601.0 percent) [04:19:59] Called DecompressByteArray: compressed_data_size=40970 data_size=246249, decompressed_data_size=246249 diff=0 [04:19:59] - Digital signature verified [04:19:59] [04:19:59] Project: 5004 (Run 6, Clone 28, Gen 35) [04:19:59] [04:19:59] Assembly optimizations on if available. [04:19:59] Entering M.D. [04:20:06] mdrun_gpu returned -1 [04:20:06] Going to send back what have done. [04:20:07] logfile size: 5502 info=5502 bed=25 hdr=1 [04:20:07] - Writing 6040 bytes of core data to disk... [04:20:07] Done: 5528 -> 2021 (compressed to 36.5 percent) [04:20:07] ... Done. [04:20:07] [04:20:07] Folding@home Core Shutdown: UNSTABLE_MACHINE [04:20:10] CoreStatus = 7A (122) [04:20:10] Sending work to server [04:20:10] - Read packet limit of 540015616... Set to 524286976. [04:20:10] + Attempting to send results [04:20:10] - Reading file work/wuresults_06.dat from core [04:20:10] (Read 2533 bytes from disk) [04:20:10] Connecting to http://171.64.65.20:8080/ [04:20:10] Posted data. [04:20:10] Initial: 0000; Conversation time very short, giving reduced weight in bandwidth avg [04:20:10] - Uploaded at ~6 kB/s [04:20:10] - Averaged speed for that direction ~4 kB/s [04:20:10] + Results successfully sent [04:20:10] Thank you for your contribution to Folding@Home. [04:20:15] Trying to send all finished work units [04:20:15] + No unsent completed units remaining. [04:20:15] - Preparing to get new work unit... [04:20:15] + Attempting to get work packet [04:20:15] - Will indicate memory of 4093 MB [04:20:15] - Connecting to assignment server [04:20:15] Connecting to http://assign-GPU.stanford.edu:8080/ [04:20:15] Posted data. [04:20:15] Initial: 40AB; - Successful: assigned to (171.64.65.20). [04:20:15] + News From Folding@Home: GPU folding beta [04:20:15] Loaded queue successfully. [04:20:15] Connecting to http://171.64.65.20:8080/ [04:20:16] Posted data. [04:20:16] Initial: 0000; - Receiving payload (expected size: 42085) [04:20:17] - Downloaded at ~41 kB/s [04:20:17] - Averaged speed for that direction ~48 kB/s [04:20:17] + Received work. [04:20:17] Trying to send all finished work units [04:20:17] + No unsent completed units remaining. [04:20:17] + Closed connections [04:20:22] [04:20:22] + Processing work unit [04:20:22] Core required: FahCore_11.exe [04:20:22] Core found. [04:20:22] Working on queue slot 07 [July 2 04:20:22] [04:20:22] + Working ... [04:20:22] - Calling 'FahCore_11.exe -dir work/ -suffix 07 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 1736 -version 612' [04:20:22] [04:20:22] *------------------------------* [04:20:22] Folding@Home GPU Core - Beta [04:20:22] Version 1.07 (Mon Jun 23 10:53:13 PDT 2008) [04:20:22] [04:20:22] Compiler : [04:20:22] Build host: amoeba [04:20:22] Preparing to commence simulation [04:20:22] - Looking at optimizations... [04:20:22] - Created dyn [04:20:22] - Files status OK [04:20:22] - Expanded 41573 -> 246249 (decompressed 592.3 percent) [04:20:22] Called DecompressByteArray: compressed_data_size=41573 data_size=246249, decompressed_data_size=246249 diff=0 [04:20:22] - Digital signature verified [04:20:22] [04:20:22] Project: 5004 (Run 3, Clone 60, Gen 24) [04:20:22] [04:20:22] Assembly optimizations on if available. [04:20:22] Entering M.D. [04:20:23] ***** Got a SIGTERM signal (2) [04:20:23] Killing all core threads Folding@Home Client Shutdown. --- Opening Log file [July 2 04:25:49] # Windows Console GPU Edition ################################################# ############################################################################### Folding@Home Client Version 6.12beta8 http://folding.stanford.edu ############################################################################### ############################################################################### Launch directory: C:\Users\Harald\AppData\Roaming\Folding@home-gpu2 Arguments: -gpu 1 -local -verbosity 9 [04:25:49] - Ask before connecting: No [04:25:49] - User name: 8800MGTX_GPU (Team 74103) [04:25:49] - User ID: 2FDC27456A21EB8 [04:25:49] - Machine ID: 7 [04:25:49] [04:25:49] Loaded queue successfully. [04:25:49] Initialization complete [04:25:49] + Benchmarking ... [04:25:49] [04:25:49] + Processing work unit [04:25:49] - Autosending finished units... [04:25:49] Trying to send all finished work units [04:25:49] + No unsent completed units remaining. [04:25:49] - Autosend completed [04:25:49] Core required: FahCore_11.exe [04:25:49] Core found. [04:25:49] Working on queue slot 07 [July 2 04:25:49] [04:25:49] + Working ... [04:25:49] - Calling 'FahCore_11.exe -dir work/ -suffix 07 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 4320 -version 612' [04:25:50] [04:25:50] *------------------------------* [04:25:50] Folding@Home GPU Core - Beta [04:25:50] Version 1.07 (Mon Jun 23 10:53:13 PDT 2008) [04:25:50] [04:25:50] Compiler : [04:25:50] Build host: amoeba [04:25:50] Preparing to commence simulation [04:25:50] - Looking at optimizations... [04:25:50] - Files status OK [04:25:50] - Expanded 41573 -> 246249 (decompressed 592.3 percent) [04:25:50] Called DecompressByteArray: compressed_data_size=41573 data_size=246249, decompressed_data_size=246249 diff=0 [04:25:50] - Digital signature verified [04:25:50] [04:25:50] Project: 5004 (Run 3, Clone 60, Gen 24) [04:25:50] [04:25:50] Assembly optimizations on if available. [04:25:50] Entering M.D. [04:25:56] mdrun_gpu returned -1 [04:25:56] Going to send back what have done. [04:25:57] logfile size: 5500 info=5500 bed=25 hdr=1 [04:25:57] - Writing 6038 bytes of core data to disk... [04:25:57] Done: 5526 -> 2014 (compressed to 36.4 percent) [04:25:57] ... Done. [04:25:57] [04:25:57] Folding@home Core Shutdown: UNSTABLE_MACHINE [04:26:00] CoreStatus = 7A (122) [04:26:00] Sending work to server [04:26:00] - Read packet limit of 540015616... Set to 524286976. [04:26:00] + Attempting to send results [04:26:00] - Reading file work/wuresults_07.dat from core [04:26:00] (Read 2526 bytes from disk) [04:26:00] Connecting to http://171.64.65.20:8080/ [04:26:00] Posted data. [04:26:00] Initial: 0000; Conversation time very short, giving reduced weight in bandwidth avg [04:26:00] - Uploaded at ~6 kB/s [04:26:00] - Averaged speed for that direction ~4 kB/s [04:26:00] + Results successfully sent [04:26:00] Thank you for your contribution to Folding@Home. [04:26:04] ***** Got a SIGTERM signal (2) [04:26:04] Killing all core threads Folding@Home Client Shutdown.

Test with -gpu 1 -local -verbosity 9 and -gpu 2 -local -verbosity 9.
Also the enviroment variables won't help.

Followed the same procedure with my 2 other machines with ATI boards, 2 clients on one card.
Copied both directories and edited the machine id with debug to 7.

my second card is connected to a HDTV with extended desktop.

What could be wrong ?

Harald
Too Cool Designs

by **skinnykid63** » Wed Jul 02, 2008 12:57 am

I'm running two 8800GT in Vista with only one display. This means I must use the override, but how?

I'm guessing the the environmental variable is the one under system properties, but what values do I enter and where (which box has what in it).

Is the name supposed to be NV_FAH_DEVICE and the value supposed to be n? And what value of n do I use. Do I need to use a path to my folders such as C:\Users\MyName\AppData\Roaming\Folding@home-gpu\NV_FAH_DEVICE=n?

by **anandhanju** » Wed Jul 02, 2008 1:11 am

DonMarkoni wrote:...
Now, seriously, this is working great!
Now, seriously, this is working great!
If You see this twice, this is realy working great!

Awesome

Thanks for the laugh!

by **seamer** » Wed Jul 02, 2008 1:17 am

OK, here's my stupid question. Is the multi-GPU client for two physical cards or two GPUs on a single card? I haven't seen a concrete answer on that one, although I may be going crazy and just not looking properly.

I have two 8600GT cards I want to use.

by **rbpeake** » Wed Jul 02, 2008 1:30 am

At some point some clearer and "simpler" directions for those of us who were not weaned on a computer but rather a human mother would be helpful!

Folding Forum

1.07 IS Multi-GPU

1.07 IS Multi-GPU

Re: 1.07 IS Multi-GPU

Re: 1.07 IS Multi-GPU

Re: 1.07 IS Multi-GPU

Re: 1.07 IS Multi-GPU

Re: 1.07 IS Multi-GPU

Re: 1.07 IS Multi-GPU

Re: 1.07 IS Multi-GPU

Re: 1.07 IS Multi-GPU

Re: 1.07 IS Multi-GPU

Re: 1.07 IS Multi-GPU

Re: 1.07 IS Multi-GPU

Re: 1.07 IS Multi-GPU

Re: 1.07 IS Multi-GPU

Re: 1.07 IS Multi-GPU

Who is online