1.07 *IS* *Multi-GPU*

Questions and comments regarding the GPU2 client running on a nVidia GPU that supports CUDA.

1.07 *IS* *Multi-GPU*

Postby slegrand » Tue Jul 01, 2008 9:06 pm

Subject says it all... I've just been given clearance to spill the beans...

But some caveats upfront..

1. Device 0 *must* be a supported CUDA and DirectX display device. It's a client quirk, and there is a workaround in development, but for now, those are the rules.

2. If all your GPUs are CUDA devices and they are all extended onto the desktop, you can just use -gpu n where n is the display device number and it will work just like an ATI client, which means follow the ATI multi-GPU FAQ and you'll be folding away.

3. In the event that 2 above is not the case, there is an override. Skip the -gpu flag and instead set the environment variable NV_FAH_DEVICE=n where n is the CUDA device you wish to use. You may find yourself also playing with CPU affinity here to get the right mix, but that's why we provided the power tools here so to speak. And also it provides a way for anyone with a tesla server/board out there to run F@H today.


3. set VAR=value is the proper syntax. If you put spaces in this, it won't work, i.e. set VAR = value is incorrect.

4. You need to set these variables either in a script that launches the client, or before you launch the client. Doing so after starting the client will have no effect until it is stopped and restarted.

I'm sure we're about to see some quirkiness, but that's what betas are all about... Have fun!
User avatar
slegrand
NVIDIA Development Team Member
 
Posts: 88
Joined: Tue Jun 10, 2008 3:49 pm

Re: 1.07 *IS* *Multi-GPU*

Postby theheyes » Tue Jul 01, 2008 9:10 pm

With two high performance GPUs sharing the same bus, are we likely to see diminishing returns adding more cards?
theheyes
 
Posts: 6
Joined: Thu Jun 19, 2008 8:27 am

Re: 1.07 *IS* *Multi-GPU*

Postby slegrand » Tue Jul 01, 2008 9:32 pm

theheyes wrote:With two high performance GPUs sharing the same bus, are we likely to see diminishing returns adding more cards?


It ought to scale like crazy... The only thing that might stop that for now is our weird desktop perf issue, but once that's cleared, the whole thing runs as a closed system on the GPU, only periodically communicating data to the CPU every 12 to 30 seconds of real-time.
User avatar
slegrand
NVIDIA Development Team Member
 
Posts: 88
Joined: Tue Jun 10, 2008 3:49 pm

Re: 1.07 *IS* *Multi-GPU*

Postby VijayPande » Tue Jul 01, 2008 9:36 pm

We'd be curious to hear what people find. I suspect that one should have one core per GPU, so that means (for most people) no more than 8 GPUs = 4 x 9800GX2 ;)
User avatar
VijayPande
Pande Group Member
 
Posts: 2651
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: 1.07 *IS* *Multi-GPU*

Postby slegrand » Tue Jul 01, 2008 9:39 pm

VijayPande wrote:We'd be curious to hear what people find. I suspect that one should have one core per GPU, so that means (for most people) no more than 8 GPUs = 4 x 9800GX2 ;)


Once we clear up what's hogging core 0, that might not even be such a limiting factor - witness the 6% CPU load on some Vista clients... Something is going horribly wrong on XP and the good part about that is that once we fix that, I suspect CPU load will be moderate.
User avatar
slegrand
NVIDIA Development Team Member
 
Posts: 88
Joined: Tue Jun 10, 2008 3:49 pm

Re: 1.07 *IS* *Multi-GPU*

Postby DonMarkoni » Tue Jul 01, 2008 9:49 pm

YOU PEOPLE ARE CRAZY! NEW CORES, NEW CLIENTS EVERY FEW HOURS! I WANT TO SLEEP! :-)
Now, seriously, this is working great!
Now, seriously, this is working great!
If You see this twice, this is realy working great!

EDIT:
First G92 GPU using one CPU core folding project 5205 R1 C286 G0 ~6000ppd
Second G92 GPU using one CPU core folding project 5002 R2 C20 G149 ~6500ppd
Only two remaining CPU cores folding project 2665 R3 C824 G23 ~2000ppd
Rig: Rampage IV Extreme, i7-3930K @ 5GHz, AData 4x2GB @ 2400MHz 9-11-9-27-1T, GTX680 Lightning @ 1200/6000
User avatar
DonMarkoni
 
Posts: 206
Joined: Mon Jun 30, 2008 6:47 pm
Location: Belgrade,Serbia

Re: 1.07 *IS* *Multi-GPU*

Postby Bernie64 » Tue Jul 01, 2008 10:24 pm

So any GX2 is seen as one GPU to the client but is physically 2 GPU's?
:?:

EDIT: 14,500ppd on one mobo?
Instinct-that which uses the other 90% of the brain conscious thought does not.
Bernie64
 
Posts: 49
Joined: Wed Jun 11, 2008 10:23 pm

Re: 1.07 *IS* *Multi-GPU*

Postby Leoslocks » Tue Jul 01, 2008 10:44 pm

If it is the machine in his signature, it is 2x 8800GTS 512MB @ 800/2000/2100

Seriously, that was really funny Don.

1.07 *IS* *Multi-GPU*
No need to change the title in the 'other thread'.
User avatar
Leoslocks
 
Posts: 389
Joined: Fri Jan 25, 2008 3:20 am

Re: 1.07 *IS* *Multi-GPU*

Postby Fadamor » Tue Jul 01, 2008 10:48 pm

Leoslocks wrote:If it is the machine in his signature, it is 2x 8800GTS 512MB @ 800/2000/2100

Seriously, that was really funny Don.

1.07 *IS* *Multi-GPU*
No need to change the title in the 'other thread'.

:P

Scott says to follow the ATI multi-gpu FAQ... I give. Where exactly is this FAQ?
Image
Image
Fadamor
 
Posts: 313
Joined: Thu Mar 13, 2008 7:21 pm
Location: Northern VA

Re: 1.07 *IS* *Multi-GPU*

Postby leexgx » Tue Jul 01, 2008 11:01 pm

cool i will test right away
Image
leexgx
 
Posts: 552
Joined: Mon Dec 03, 2007 8:05 am

Re: 1.07 *IS* *Multi-GPU*

Postby cbr1100xxVanish » Tue Jul 01, 2008 11:03 pm

It doesn't work for me.

Vista 64 / 1.07 / 177.35 / 2 x 8800M GTX.
One card works but the second gives unstable problems starting with a "mdrun_gpu returned -1"

Code: Select all
--- Opening Log file [July 2 04:19:57]


# Windows Console GPU Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.12beta8

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Users\Harald\AppData\Roaming\Folding@home-gpu2
Arguments: -gpu 2 -local -verbosity 9

[04:19:57] - Ask before connecting: No
[04:19:57] - User name: 8800MGTX_GPU (Team 74103)
[04:19:57] - User ID: 2FDC27456A21EB8
[04:19:57] - Machine ID: 7
[04:19:57]
[04:19:57] Loaded queue successfully.
[04:19:57] Initialization complete
[04:19:57] + Benchmarking ...
[04:19:57] - Preparing to get new work unit...
[04:19:57] - Autosending finished units...
[04:19:57] + Attempting to get work packet
[04:19:57] Trying to send all finished work units
[04:19:57] + No unsent completed units remaining.
[04:19:57] - Autosend completed
[04:19:57] - Will indicate memory of 4093 MB
[04:19:57] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 15, Stepping: 11
[04:19:57] - Connecting to assignment server
[04:19:57] Connecting to http://assign-GPU.stanford.edu:8080/
[04:19:58] Posted data.
[04:19:58] Initial: 40AB; - Successful: assigned to (171.64.65.20).
[04:19:58] + News From Folding@Home: GPU folding beta
[04:19:58] Loaded queue successfully.
[04:19:58] Connecting to http://171.64.65.20:8080/
[04:19:59] Posted data.
[04:19:59] Initial: 0000; - Receiving payload (expected size: 41482)
[04:19:59] Conversation time very short, giving reduced weight in bandwidth avg
[04:19:59] - Downloaded at ~81 kB/s
[04:19:59] - Averaged speed for that direction ~50 kB/s
[04:19:59] + Received work.
[04:19:59] + Closed connections
[04:19:59]
[04:19:59] + Processing work unit
[04:19:59] Core required: FahCore_11.exe
[04:19:59] Core found.
[04:19:59] Working on queue slot 06 [July 2 04:19:59]
[04:19:59] + Working ...
[04:19:59] - Calling 'FahCore_11.exe -dir work/ -suffix 06 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 1736 -version 612'

[04:19:59]
[04:19:59] *------------------------------*
[04:19:59] Folding@Home GPU Core - Beta
[04:19:59] Version 1.07 (Mon Jun 23 10:53:13 PDT 2008)
[04:19:59]
[04:19:59] Compiler  :
[04:19:59] Build host: amoeba
[04:19:59] Preparing to commence simulation
[04:19:59] - Looking at optimizations...
[04:19:59] - Created dyn
[04:19:59] - Files status OK
[04:19:59] - Expanded 40970 -> 246249 (decompressed 601.0 percent)
[04:19:59] Called DecompressByteArray: compressed_data_size=40970 data_size=246249, decompressed_data_size=246249 diff=0
[04:19:59] - Digital signature verified
[04:19:59]
[04:19:59] Project: 5004 (Run 6, Clone 28, Gen 35)
[04:19:59]
[04:19:59] Assembly optimizations on if available.
[04:19:59] Entering M.D.
[04:20:06] mdrun_gpu returned -1
[04:20:06] Going to send back what have done.
[04:20:07] logfile size: 5502 info=5502 bed=25 hdr=1
[04:20:07] - Writing 6040 bytes of core data to disk...
[04:20:07] Done: 5528 -> 2021 (compressed to 36.5 percent)
[04:20:07]   ... Done.
[04:20:07]
[04:20:07] Folding@home Core Shutdown: UNSTABLE_MACHINE
[04:20:10] CoreStatus = 7A (122)
[04:20:10] Sending work to server
[04:20:10] - Read packet limit of 540015616... Set to 524286976.


[04:20:10] + Attempting to send results
[04:20:10] - Reading file work/wuresults_06.dat from core
[04:20:10]   (Read 2533 bytes from disk)
[04:20:10] Connecting to http://171.64.65.20:8080/
[04:20:10] Posted data.
[04:20:10] Initial: 0000; Conversation time very short, giving reduced weight in bandwidth avg
[04:20:10] - Uploaded at ~6 kB/s
[04:20:10] - Averaged speed for that direction ~4 kB/s
[04:20:10] + Results successfully sent
[04:20:10] Thank you for your contribution to Folding@Home.
[04:20:15] Trying to send all finished work units
[04:20:15] + No unsent completed units remaining.
[04:20:15] - Preparing to get new work unit...
[04:20:15] + Attempting to get work packet
[04:20:15] - Will indicate memory of 4093 MB
[04:20:15] - Connecting to assignment server
[04:20:15] Connecting to http://assign-GPU.stanford.edu:8080/
[04:20:15] Posted data.
[04:20:15] Initial: 40AB; - Successful: assigned to (171.64.65.20).
[04:20:15] + News From Folding@Home: GPU folding beta
[04:20:15] Loaded queue successfully.
[04:20:15] Connecting to http://171.64.65.20:8080/
[04:20:16] Posted data.
[04:20:16] Initial: 0000; - Receiving payload (expected size: 42085)
[04:20:17] - Downloaded at ~41 kB/s
[04:20:17] - Averaged speed for that direction ~48 kB/s
[04:20:17] + Received work.
[04:20:17] Trying to send all finished work units
[04:20:17] + No unsent completed units remaining.
[04:20:17] + Closed connections
[04:20:22]
[04:20:22] + Processing work unit
[04:20:22] Core required: FahCore_11.exe
[04:20:22] Core found.
[04:20:22] Working on queue slot 07 [July 2 04:20:22]
[04:20:22] + Working ...
[04:20:22] - Calling 'FahCore_11.exe -dir work/ -suffix 07 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 1736 -version 612'

[04:20:22]
[04:20:22] *------------------------------*
[04:20:22] Folding@Home GPU Core - Beta
[04:20:22] Version 1.07 (Mon Jun 23 10:53:13 PDT 2008)
[04:20:22]
[04:20:22] Compiler  :
[04:20:22] Build host: amoeba
[04:20:22] Preparing to commence simulation
[04:20:22] - Looking at optimizations...
[04:20:22] - Created dyn
[04:20:22] - Files status OK
[04:20:22] - Expanded 41573 -> 246249 (decompressed 592.3 percent)
[04:20:22] Called DecompressByteArray: compressed_data_size=41573 data_size=246249, decompressed_data_size=246249 diff=0
[04:20:22] - Digital signature verified
[04:20:22]
[04:20:22] Project: 5004 (Run 3, Clone 60, Gen 24)
[04:20:22]
[04:20:22] Assembly optimizations on if available.
[04:20:22] Entering M.D.
[04:20:23] ***** Got a SIGTERM signal (2)
[04:20:23] Killing all core threads

Folding@Home Client Shutdown.


--- Opening Log file [July 2 04:25:49]


# Windows Console GPU Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.12beta8

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Users\Harald\AppData\Roaming\Folding@home-gpu2
Arguments: -gpu 1 -local -verbosity 9

[04:25:49] - Ask before connecting: No
[04:25:49] - User name: 8800MGTX_GPU (Team 74103)
[04:25:49] - User ID: 2FDC27456A21EB8
[04:25:49] - Machine ID: 7
[04:25:49]
[04:25:49] Loaded queue successfully.
[04:25:49] Initialization complete
[04:25:49] + Benchmarking ...
[04:25:49]
[04:25:49] + Processing work unit
[04:25:49] - Autosending finished units...
[04:25:49] Trying to send all finished work units
[04:25:49] + No unsent completed units remaining.
[04:25:49] - Autosend completed
[04:25:49] Core required: FahCore_11.exe
[04:25:49] Core found.
[04:25:49] Working on queue slot 07 [July 2 04:25:49]
[04:25:49] + Working ...
[04:25:49] - Calling 'FahCore_11.exe -dir work/ -suffix 07 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 4320 -version 612'

[04:25:50]
[04:25:50] *------------------------------*
[04:25:50] Folding@Home GPU Core - Beta
[04:25:50] Version 1.07 (Mon Jun 23 10:53:13 PDT 2008)
[04:25:50]
[04:25:50] Compiler  :
[04:25:50] Build host: amoeba
[04:25:50] Preparing to commence simulation
[04:25:50] - Looking at optimizations...
[04:25:50] - Files status OK
[04:25:50] - Expanded 41573 -> 246249 (decompressed 592.3 percent)
[04:25:50] Called DecompressByteArray: compressed_data_size=41573 data_size=246249, decompressed_data_size=246249 diff=0
[04:25:50] - Digital signature verified
[04:25:50]
[04:25:50] Project: 5004 (Run 3, Clone 60, Gen 24)
[04:25:50]
[04:25:50] Assembly optimizations on if available.
[04:25:50] Entering M.D.
[04:25:56] mdrun_gpu returned -1
[04:25:56] Going to send back what have done.
[04:25:57] logfile size: 5500 info=5500 bed=25 hdr=1
[04:25:57] - Writing 6038 bytes of core data to disk...
[04:25:57] Done: 5526 -> 2014 (compressed to 36.4 percent)
[04:25:57]   ... Done.
[04:25:57]
[04:25:57] Folding@home Core Shutdown: UNSTABLE_MACHINE
[04:26:00] CoreStatus = 7A (122)
[04:26:00] Sending work to server
[04:26:00] - Read packet limit of 540015616... Set to 524286976.


[04:26:00] + Attempting to send results
[04:26:00] - Reading file work/wuresults_07.dat from core
[04:26:00]   (Read 2526 bytes from disk)
[04:26:00] Connecting to http://171.64.65.20:8080/
[04:26:00] Posted data.
[04:26:00] Initial: 0000; Conversation time very short, giving reduced weight in bandwidth avg
[04:26:00] - Uploaded at ~6 kB/s
[04:26:00] - Averaged speed for that direction ~4 kB/s
[04:26:00] + Results successfully sent
[04:26:00] Thank you for your contribution to Folding@Home.
[04:26:04] ***** Got a SIGTERM signal (2)
[04:26:04] Killing all core threads

Folding@Home Client Shutdown.


Test with -gpu 1 -local -verbosity 9 and -gpu 2 -local -verbosity 9.
Also the enviroment variables won't help.

Followed the same procedure with my 2 other machines with ATI boards, 2 clients on one card.
Copied both directories and edited the machine id with debug to 7.

my second card is connected to a HDTV with extended desktop.

What could be wrong ?

Harald
Too Cool Designs
Last edited by cbr1100xxVanish on Wed Jul 02, 2008 4:34 am, edited 1 time in total.
Harald,
Too Cool Designs.

Win2003 server, XP 32, Vista 32, Vista 64
SMP, GPU2 Nvidia, GPU2 Ati, PS3
9800GTX+_OC,9800GTX, 8800MGTX, 8700MGT, Q2700M, Q1700, Q140M, HD3870,HD3870_OC
X9100, Q6700, E8500, E6750, Xeon, AMD X2 6400+
User avatar
cbr1100xxVanish
 
Posts: 63
Joined: Sat May 24, 2008 9:44 am

Re: 1.07 *IS* *Multi-GPU*

Postby skinnykid63 » Tue Jul 01, 2008 11:57 pm

I'm running two 8800GT in Vista with only one display. This means I must use the override, but how?

I'm guessing the the environmental variable is the one under system properties, but what values do I enter and where (which box has what in it).

Is the name supposed to be NV_FAH_DEVICE and the value supposed to be n? And what value of n do I use. Do I need to use a path to my folders such as C:\Users\MyName\AppData\Roaming\Folding@home-gpu\NV_FAH_DEVICE=n?
skinnykid63
 
Posts: 23
Joined: Mon Jun 23, 2008 2:11 pm

Re: 1.07 *IS* *Multi-GPU*

Postby anandhanju » Wed Jul 02, 2008 12:11 am

DonMarkoni wrote:...
Now, seriously, this is working great!
Now, seriously, this is working great!
If You see this twice, this is realy working great!
Awesome :lol: Thanks for the laugh!
anandhanju
 
Posts: 660
Joined: Mon Dec 03, 2007 4:33 am

Re: 1.07 *IS* *Multi-GPU*

Postby seamer » Wed Jul 02, 2008 12:17 am

OK, here's my stupid question. Is the multi-GPU client for two physical cards or two GPUs on a single card? I haven't seen a concrete answer on that one, although I may be going crazy and just not looking properly.

I have two 8600GT cards I want to use.
seamer
 
Posts: 4
Joined: Wed Jul 02, 2008 12:15 am

Re: 1.07 *IS* *Multi-GPU*

Postby rbpeake » Wed Jul 02, 2008 12:30 am

At some point some clearer and "simpler" directions for those of us who were not weaned on a computer but rather a human mother would be helpful! ;)
rbpeake
 
Posts: 320
Joined: Sun Jun 15, 2008 4:39 pm
Location: NYC Metro Area

Next

Return to Windows GPU2 (nVidia GPUs)

Who is online

Users browsing this forum: No registered users and 1 guest

cron