Configuring Multiple/Additional GPUs [Continuous Giude]

Moderators: Site Moderators, PandeGroup

Re: Configuring new GPUs

Postby Flaschie » Thu May 15, 2014 8:44 pm

I think I have encounted the same problem as you. I have two GPUs in one PC, an internal GPU and a dedicated GPU. I was not interested in folding on the internal GPU, so I created only one slot and set the GPU-index to match my dedicated GPU. However, every time I restarted my PC, the configuration got messed up and in-progress WUs got dumped (just by starting FAH this happened). My solution was to have two slots, one for each GPU, set up the OpenCL index to match the card I wanted (this got switched, so in my case I had to manually set it to "1" for the dedicated), and added "pause-on-start" flag on the internal GPU-slot. This way, it will never fold on the internal GPU unless I manually start that one, so basically giving me the setup that I want.

This was btw a clean install with both GPUs present..... And I also have a similar setup in another computer, but there my dedicated GPU is listed as the "GPU 0" in the system config. This gave me no problem at all! So maybe the problem is related to trying to make the "GPU 1" to be only slot?
Flaschie
 
Posts: 30
Joined: Sun Mar 11, 2012 5:52 pm

Re: Configuring new GPUs

Postby bruce » Thu May 15, 2014 9:40 pm

There may be two problems being discussed here.

If you look at V7's System Configuration you can see various things. This one says
GPUs: 1
GPU 0: NVIDIA ...(etc.)
CUDA: 3.0

This is simple and, of course, always works. If I try to set gpu-index to anything other than 0, it will revert to -1 and effectively to zero. If I have another slot (say for the CPU) it doesn't matter which order the slots happen to be configured.

On another system it says
GPUs: 2
GPU 0: UNSUPPORTED ... (etc.)
GPU 1: ATI .... (etc.)
CUDA: Not Detected

(GPU 0 is and old motherboard-embedded GPU and can't fold). V7 believes that only GPU 1 exists for the purpose of folding. If I choose to disable the unsupported GPU, I will have MOVED the ATI GPU to gpu-index 0 but that might or might not be a different slot.

On another system, I have
GPUs: 2
GPU 0: NVIDIA ... (etc.)
GPU 1: NVIDIA ... (etc.)
CUDA: 3.0

where one is a Fermi and the other is a Tesla. If I had added these GPUs one-at-a-time, V7 would have configured whichever was installed first and started folding on it. If I later added the second GPU, I can no longer predict which one will be detected first. V7 might choose the same GPU first, in which case the active WU would still match the GPU. A slot would be added for the second GPU and a WU downloaded for it and everything would be fine.

If, on the other hand V7 detects the new GPU first, associates it with the first slot, the existing WU won't be matched to the previous (first and only) GPU. The WU won't run. A new slot will be added for the GPU that is detected second, even though it was previously the only GPU and therefore associated with the WU that's in the first slot. As far as the software is concerned, you MOVED the original GPU to make room for a different type of GPU, thereby messing things up.

It should be noted that "MOVED" is not a physical location associated with a specific slot on the motherboard but rather the order that lspci detects them.
bruce
 
Posts: 20828
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Configuring new GPUs

Postby DownshiftArtist » Fri May 16, 2014 12:58 pm

bruce wrote:The GPUs must match the WUs that have been downloaded. Under normal circumstances, if you uninstall and reinstall everything, including any unfinsihed WUs (best to FINISH them first) there's nothing wrong with the slots being reconfigured in the opposite order. The freshly downloaded WUs will then match whichever GPU has been configured for that slot.

In other words, the only circumstances where I've seen this sort of thing with the latest client is when a second GPU is added after the first one is running. This has a 50% chance of messing up the order that V7 would have chosen if it had set up a fresh configuration with no history.

I don't remember seeing a warning about this and we should probably have one somewhere saying that the FAH client must have a clean reinstal if GPUs are added or replaced with a different type of device. The fundamental problem is that wherever that warning is placed, you wouldn't have seen it.


I did a clean re-install and everything but the client still couldn't detect them properly and, as stated before, I can configure them properly but then if I quit/restart the client for any reason, the client would default back to the wrong card every time, no matter what....that was until I tried P5's idea this morning of setting up a slot for each card and just setting the one I don't want to use to pause. Worked like a charm! :mrgreen:

So the fix for this problem:

1. Make sure to do a clean removal and re-install first to see if this will fix the issue. If not proceed to step 2.

2. Install GPU-Z and make a note of the order of the GPUs in the drop down list at the bottom, 0 for the first GPU, 1 for the second and so on; this can be 'List A'

3. Use command prompt (Start button, 'cmd' in the search bar and press enter) to run C:\Program Files (x86)\FAHClient\fachclient.exe --lspci and note the order the GPUs come up on this list; 'List B'

4. Open F@H Client and clear all the automatically assigned slots for the GPUs

5. Create new GPU slots and set the each GPU index according to "list B', and the core indices according to 'list A'. Do this for all GPUs present
Ex. gpu-index = 0, CUDA-index = 1, OpenCL-index = 1 will fold on the second card on the GPU-Z list.

6. For any GPU you DO NOT want to fold, under 'Extra Slot Options' add Name: paused Value: true

7. Save your configuration then quit and restart your client to verify that config has been saved properly.

8. FOLD!!



Thanks to P5, 7im, Flashie & Bruce for all the help!
DownshiftArtist
 
Posts: 37
Joined: Mon Apr 01, 2013 6:41 pm

Re: Configuring new GPUs [Solved]

Postby bruce » Fri May 16, 2014 7:50 pm

I'm glad that method worked for you. We do need to improve that "how-to" slightly, so any information you can provide may help, especially if you have a mixture of detected GPUs.

Please post the lspci output as well as lists A and B plus your final config.xml. Some 35 lines from the top of the log, you should also see the detected GPUs in a format somewhat like the following (You probably need to do a Refresh if you're looking at the log FAHControl, but it's also in the System Info panel.)

GPUs: 3
GPU 0: UNSUPPORTED ....
GPU 1: NVIDIA:2 ...
GPU 2: ATI:5....
CUDA: [Not Detected] or a version number.

Does this order match list B

Unfortunately, CUDA-Index and OpenCL-Index do not always match.
bruce
 
Posts: 20828
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Configuring new GPUs [Solved]

Postby P5-133XL » Fri May 16, 2014 8:08 pm

GPU-Z is not a reliable tool for determining the order of GPU's. Much better is FAHBench primarily because it can be used to get cuda-index and opencl-index numbers.
Image
P5-133XL
 
Posts: 4034
Joined: Sun Dec 02, 2007 4:36 am
Location: Salem. OR USA

Re: Configuring new GPUs [Solved]

Postby 7im » Fri May 16, 2014 10:15 pm

Have to be careful with FAHBench as well. OpenCL list can show CPUs in the list.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
User avatar
7im
 
Posts: 14648
Joined: Thu Nov 29, 2007 4:30 pm
Location: Arizona

Re: Configuring new GPUs [Solved]

Postby DownshiftArtist » Mon May 19, 2014 11:55 pm

UPDATED WALKTHROUGH

1. Make sure to do a clean removal and re-install first to see if this will fix the issue. If not proceed to step 2.

2. Install GPU-Z and make a note of the order of the GPUs in the drop down list at the bottom, 0 for the first GPU, 1 for the second and so on; this can be 'List A'

GPU-Z, for "List A". GTX 570 will be 0, and the 260 will be 1

Image

3. Use command prompt (Start button, 'cmd' in the search bar and press enter) to run C:\Program Files (x86)\FAHClient\fachclient.exe --lspci and note the order the GPUs come up on this list; 'List B'


C:\Program Files (x86)\FAHClient\fachclient.exe --lspci
Code: Select all
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\Family>cd\

C:\>cd program files (x86)

C:\Program Files (x86)>cd fahclient

C:\Program Files (x86)\FAHClient>fahclient.exe --lspci
VendorID:DeviceID:Vendor Name:Description
0x104c:0x8023:Texas Instruments:Texas Instruments 1394 OHCI Compliant Host Contr
oller
0x1095:0x3114:Silicon Image, Inc.:Silicon Image SiI 3114 SoftRaid 5 Controller
0x10de:0x0360:NVIDIA Corporation:PCI standard ISA bridge
0x10de:0x0368:NVIDIA Corporation:NVIDIA nForce PCI System Management
0x10de:0x0369:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x036c:NVIDIA Corporation:Standard OpenHCD USB Host Controller
0x10de:0x036d:NVIDIA Corporation:Standard Enhanced PCI to USB Host Controller
0x10de:0x036e:NVIDIA Corporation:Standard Dual Channel PCI IDE Controller
0x10de:0x0370:NVIDIA Corporation:PCI standard PCI-to-PCI bridge
0x10de:0x0371:NVIDIA Corporation:High Definition Audio Controller
0x10de:0x0373:NVIDIA Corporation:NVIDIA nForce Networking Controller
0x10de:0x0373:NVIDIA Corporation:NVIDIA nForce Networking Controller
0x10de:0x0376:NVIDIA Corporation:PCI Express standard Root Port
0x10de:0x037f:NVIDIA Corporation:NVIDIA nForce Serial ATA Controller
0x10de:0x037f:NVIDIA Corporation:NVIDIA nForce Serial ATA Controller
0x10de:0x037f:NVIDIA Corporation:NVIDIA nForce Serial ATA Controller
0x10de:0x03a1:NVIDIA Corporation:PCI standard host CPU bridge
0x10de:0x03a8:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03a9:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03aa:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03ab:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03ac:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03ad:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03ae:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03af:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03b0:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03b1:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03b2:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03b3:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03b4:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03b5:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03b6:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03b7:NVIDIA Corporation:PCI Express standard Root Port
0x10de:0x03ba:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x03bc:NVIDIA Corporation:PCI standard RAM Controller
0x10de:0x05b1:NVIDIA Corporation:PCI Express standard Downstream Switch Port
0x10de:0x05b1:NVIDIA Corporation:PCI Express standard Downstream Switch Port
0x10de:0x05b1:NVIDIA Corporation:PCI Express standard Downstream Switch Port
0x10de:0x05e2:NVIDIA Corporation:GT200 [GeForce GTX 260]
0x10de:0x0e09:NVIDIA Corporation:High Definition Audio Controller
0x10de:0x1081:NVIDIA Corporation:GF110 [GeForce GTX 570]

C:\Program Files (x86)\FAHClient>


0x10de:0x05e2:NVIDIA Corporation:GT200 [GeForce GTX 260]
0x10de:0x0e09:NVIDIA Corporation:High Definition Audio Controller
0x10de:0x1081:NVIDIA Corporation:GF110 [GeForce GTX 570]


This is what you're looking for, and this will be "List B". In this case, the 260 will be 0 and the 570 will be 1

4. Open F@H Client and clear all the automatically assigned slots for the GPUs

5. Create new GPU slots and set the each GPU index according to "list B', and the core indices according to 'list A'. Do this for all GPUs present
Ex. gpu-index = 0, CUDA-index = 1, OpenCL-index = 1 will fold on GTX 260

Image

6. For any GPU you DO NOT want to fold, under 'Extra Slot Options' add Name: paused Value: true

7. Save your configuration then quit and restart your client to verify that config has been saved properly.

After you set up the GPU configs correctly, the config file should look something like this

Code: Select all
23:37:52:<config>
23:37:52:  <!-- Network -->
23:37:52:  <proxy v=':8080'/>
23:37:52:
23:37:52:  <!-- Slot Control -->
23:37:52:  <power v='full'/>
23:37:52:
23:37:52:  <!-- User Information -->
23:37:52:  <passkey v='********************************'/>
23:37:52:  <team v='37726'/>
23:37:52:  <user v='DownshiftArtist'/>
23:37:52:
23:37:52:  <!-- Folding Slots -->
23:37:52:  <slot id='0' type='GPU'>
23:37:52:    <cuda-index v='1'/>
23:37:52:    <gpu-index v='0'/>
23:37:52:    <opencl-index v='1'/>
23:37:52:    <paused v='true'/>
23:37:52:  </slot>
23:37:52:  <slot id='2' type='GPU'>
23:37:52:    <client-type v='beta'/>
23:37:52:    <cuda-index v='0'/>
23:37:52:    <gpu-index v='1'/>
23:37:52:    <opencl-index v='0'/>
23:37:52:  </slot>
23:37:52:</config>


edit: NOTE: Your slot id may be different as it is based on whatever number your client assigns to it.


8. FOLD!!
DownshiftArtist
 
Posts: 37
Joined: Mon Apr 01, 2013 6:41 pm

Re: Configuring Multiple/Additional GPUs [Solved]

Postby bruce » Tue May 20, 2014 3:48 pm

Very nice write-up, DownshiftArtist. Thanks.

I would add

8a. If there are any GPU WUs have accidentally been downloaded, pause FAH and delete those WUs. [With proper care, this should have been avoided from the beginning.]
8b. FOLD!!
bruce
 
Posts: 20828
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Configuring Multiple/Additional GPUs [Solved]

Postby 7im » Tue May 20, 2014 5:40 pm

As I said before, this is a general guideline. You CANNOT assume that CUDA and OPENCL indexes are ALWAYS the same number. They are NOT always going to match the one value shown in List A. It just happens to be the same for YOUR setup.
User avatar
7im
 
Posts: 14648
Joined: Thu Nov 29, 2007 4:30 pm
Location: Arizona

Re: Configuring Multiple/Additional GPUs [Solved]

Postby DownshiftArtist » Tue May 20, 2014 6:44 pm

7im wrote:As I said before, this is a general guideline. You CANNOT assume that CUDA and OPENCL indexes are ALWAYS the same number. They are NOT always going to match the one value shown in List A. It just happens to be the same for YOUR setup.



Ok, then in what scenario would they be different? / What would make them different? Please, let me know so that I may update the guide appropriately.
DownshiftArtist
 
Posts: 37
Joined: Mon Apr 01, 2013 6:41 pm

Re: Configuring Multiple/Additional GPUs [Solved]

Postby 7im » Tue May 20, 2014 7:24 pm

When you have an OpenCL device that is not a CUDA device.

One example is when you two GPUs, one from NV and one from AMD.
User avatar
7im
 
Posts: 14648
Joined: Thu Nov 29, 2007 4:30 pm
Location: Arizona

Re: Configuring Multiple/Additional GPUs [Solved]

Postby P5-133XL » Tue May 20, 2014 7:38 pm

Another example would be if you had an OpenCL driver for your CPU (Cuda does not have that capability) then the CUDA and OpenCL numbers will be different.
P5-133XL
 
Posts: 4034
Joined: Sun Dec 02, 2007 4:36 am
Location: Salem. OR USA

Re: Configuring Multiple/Additional GPUs [Solved]

Postby bruce » Tue May 20, 2014 8:32 pm

By far the best way that I've seen to figure out how to set these is with GPUBench. You'll have to play around with the command line parameters to get all the information, but it does work. Basically there are two sets of drivers, one for CUDA and one for OpenCL, and you have to query them separately. When you query either OpenCL or CUDA, it will list the devices recognized and their indices. All CUDA devices are sequential, (without skipping a spot for an OpenCL device if you have an ATI). All OpenCL devices are sequential, whether they're supported by FAH or not.
bruce
 
Posts: 20828
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Configuring Multiple/Additional GPUs [Solved]

Postby DownshiftArtist » Wed May 21, 2014 12:29 am

Do any of you have a set up like any of those situation that could lend a hand re-working this write up? Would the indices ever be a different number such as 2 or 3? Also, Bruce, would you be able to provide a screen shot of how the GPU bench shows indices? Please understand that I was only trying to be helpful and, since there are other variables that I did not consider, I would really appreciate any help getting this right. There's enough misinformation on the internet and I'd rather not contribute to it.
DownshiftArtist
 
Posts: 37
Joined: Mon Apr 01, 2013 6:41 pm

Re: Configuring Multiple/Additional GPUs [Solved]

Postby bruce » Wed May 21, 2014 4:38 am

I didn't mean it as a criticism. The writeup has never been perfect and it has been improved several times (including by you).

To come up with a good example, we need to find somebody who combines GPUs from multiple vendors who is willing to capture their data. Recently I've been adding systems, not GPUs so my GPUs are now one-per-system.
bruce
 
Posts: 20828
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

PreviousNext

Return to V7.4.4 Public Release Windows/Linux/MacOS X

Who is online

Users browsing this forum: No registered users and 0 guests

cron