Why are projects 9415 and 9414 such low PPD?

If you think it might be a driver problem, see viewforum.php?f=79

Moderators: Site Moderators, FAHC Science Team

Luscious
Posts: 49
Joined: Sat Oct 13, 2012 6:38 am

Why are projects 9415 and 9414 such low PPD?

Post by Luscious »

I have noticed over the last four-eight weeks that my overall PPD has been declining, while my WU count has been increasing. Examining this closer I notice a heavy workload of projects 9414 and 9415 on my system. Other GPU projects are out there that give much higher PPD, but for some reason I seem to be getting work units from these two projects almost exclusively.

My cause preference is set to any and hasn't been changed.

My cards (GTX980Ti) can push 600K PPD on a good day, yet these two projects limit me to 390-400K only.

Curious if Pascal cards are showing the same low PPD on these projects.
bollix47
Posts: 2941
Joined: Sun Dec 02, 2007 5:04 am
Location: Canada

Re: Why are projects 9415 and 9414 such low PPD?

Post by bollix47 »

While there are some projects with lower atom counts that don't do as well as other projects on the high end GPUs these two projects seem to be fine with my 980ti, both getting around 640K PPD. With the lower atom count there will be more frequent traffic between the GPU and the CPU.

A few things to look at:

1. Temperatures of both the GPU & the CPU ... i.e. is there throttling happening?
2. Have you allowed a full CPU core for each GPU?
3. Power availability ... is there enough?
4. How much system RAM is available for the folding process?

You could post the System Info and Config sections of your log.txt file and we may be able to spot something that requires adjustment.

If you're unsure how to post that info please see this thread.
Luscious
Posts: 49
Joined: Sat Oct 13, 2012 6:38 am

Re: Why are projects 9415 and 9414 such low PPD?

Post by Luscious »

A few things to look at:

1. Temperatures of both the GPU & the CPU ... i.e. is there throttling happening?
2. Have you allowed a full CPU core for each GPU?
3. Power availability ... is there enough?
4. How much system RAM is available for the folding process?
1. CPU hovers around 56C while the GPU's are between 40-42C. It's a well-cooled system with no throttling.
2. Pretty sure it's turned on. How do I check?
3. 1600W PSU pulling 1050W from the wall when under load.
4. 32GB installed with each x21 core using about 117MB according to task manager

Here is the log snippet with system info and config:

Code: Select all

*********************** Log Started 2017-11-02T03:51:20Z ***********************
03:51:20:************************* Folding@home Client *************************
03:51:20:      Website: http://folding.stanford.edu/
03:51:20:    Copyright: (c) 2009-2014 Stanford University
03:51:20:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
03:51:20:         Args: 
03:51:20:       Config: C:/ProgramData/FAHClient/config.xml
03:51:20:******************************** Build ********************************
03:51:20:      Version: 7.4.4
03:51:20:         Date: Mar 4 2014
03:51:20:         Time: 20:26:54
03:51:20:      SVN Rev: 4130
03:51:20:       Branch: fah/trunk/client
03:51:20:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
03:51:20:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
03:51:20:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
03:51:20:     Platform: win32 XP
03:51:20:         Bits: 32
03:51:20:         Mode: Release
03:51:20:******************************* System ********************************
03:51:20:          CPU: Intel(R) Core(TM) i7-5960X CPU @ 3.00GHz
03:51:20:       CPU ID: GenuineIntel Family 6 Model 63 Stepping 2
03:51:20:         CPUs: 16
03:51:20:       Memory: 23.87GiB
03:51:20:  Free Memory: 18.03GiB
03:51:20:      Threads: WINDOWS_THREADS
03:51:20:   OS Version: 6.1
03:51:20:  Has Battery: true
03:51:20:   On Battery: false
03:51:20:   UTC Offset: -7
03:51:20:          PID: 3956
03:51:20:          CWD: C:/ProgramData/FAHClient
03:51:20:           OS: Windows 7 Professional
03:51:20:      OS Arch: AMD64
03:51:20:         GPUs: 4
03:51:20:        GPU 0: NVIDIA:7 GM200 [GeForce GTX 980 Ti] 5632
03:51:20:        GPU 1: NVIDIA:7 GM200 [GeForce GTX 980 Ti] 5632
03:51:20:        GPU 2: NVIDIA:7 GM200 [GeForce GTX 980 Ti] 5632
03:51:20:        GPU 3: NVIDIA:7 GM200 [GeForce GTX 980 Ti] 5632
03:51:20:         CUDA: 5.2
03:51:20:  CUDA Driver: 8000
03:51:20:Win32 Service: false
03:51:20:***********************************************************************
03:51:20:<config>
03:51:20:  <!-- Network -->
03:51:20:  <proxy v=':8080'/>
03:51:20:
03:51:20:  <!-- Slot Control -->
03:51:20:  <power v='full'/>
03:51:20:
03:51:20:  <!-- User Information -->
03:51:20:  <passkey v='********************************'/>
03:51:20:  <team v='155369'/>
03:51:20:  <user v='Luscious'/>
03:51:20:
03:51:20:  <!-- Folding Slots -->
03:51:20:  <slot id='0' type='CPU'>
03:51:20:    <client-type v='advanced'/>
03:51:20:    <cpus v='8'/>
03:51:20:  </slot>
03:51:20:  <slot id='1' type='GPU'>
03:51:20:    <client-type v='advanced'/>
03:51:20:    <max-packet-size v='big'/>
03:51:20:  </slot>
03:51:20:  <slot id='2' type='GPU'>
03:51:20:    <client-type v='advanced'/>
03:51:20:    <max-packet-size v='big'/>
03:51:20:  </slot>
03:51:20:  <slot id='3' type='GPU'>
03:51:20:    <client-type v='advanced'/>
03:51:20:    <max-packet-size v='big'/>
03:51:20:  </slot>
03:51:20:  <slot id='4' type='GPU'>
03:51:20:    <client-type v='advanced'/>
03:51:20:    <max-packet-size v='big'/>
03:51:20:  </slot>
03:51:20:</config>
Something does sound quite amiss if your 980TI is doing 640K on these two projects. Which GeForce driver are you using?
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Why are projects 9415 and 9414 such low PPD?

Post by foldy »

You could try to disable the CPU slot to see if GPU PPD goes up for these projects.
bollix47
Posts: 2941
Joined: Sun Dec 02, 2007 5:04 am
Location: Canada

Re: Why are projects 9415 and 9414 such low PPD?

Post by bollix47 »

4. 32GB installed with each x21 core using about 117MB according to task manager
03:51:20: Memory: 23.87GiB
03:51:20: Free Memory: 18.03GiB
Something doesn't look right there. I have both linux and Win systems and the difference between actual memory and what is shown in my log is very small compared to 32 vs 23. Have you run a memory test or is your actual memory 24GB?

In my case I'm using linux drivers 370.28 and I know linux does get a bit more PPD than Windows but the difference should not be as large as what you're seeing.

As foldy suggests try pausing the CPU slot next time you get one of these projects just to see if there's a difference.
2. Pretty sure it's turned on. How do I check?
According to your log you have 16 threads and you're only using 8 for the CPU slot leaving 8 for the 4 GPUs. That should be plenty.

Another area to look into is your motherboard. Are your pci-e slots all running at x16 when 4 of them are being used? What is their bandwidth?
Kjetil
Posts: 178
Joined: Sat Apr 14, 2012 5:56 pm
Location: Stavanger Norway

Re: Why are projects 9415 and 9414 such low PPD?

Post by Kjetil »

My ppd is 900k(p13147) and 1.5k on (p13200). Evga 1080Ti sc2 at 2050.
Nathan_P
Posts: 1180
Joined: Wed Apr 01, 2009 9:22 pm
Hardware configuration: Asus Z8NA D6C, 2 x5670@3.2 Ghz, , 12gb Ram, GTX 980ti, AX650 PSU, win 10 (daily use)

Asus Z87 WS, Xeon E3-1230L v3, 8gb ram, KFA GTX 1080, EVGA 750ti , AX760 PSU, Mint 18.2 OS

Not currently folding
Asus Z9PE- D8 WS, 2 E5-2665@2.3 Ghz, 16Gb 1.35v Ram, Ubuntu (Fold only)
Asus Z9PA, 2 Ivy 12 core, 16gb Ram, H folding appliance (fold only)
Location: Jersey, Channel islands

Re: Why are projects 9415 and 9414 such low PPD?

Post by Nathan_P »

There is nothing wrong with the PPD on 9414 and 9415 on pascal, on a 1080 I am getting over 900k PPD and nearly 700k ppd on a 1070. My 980ti used to manage anything from 385k to 590k PPD but i'm not sure what the project spread was.
Image
Luscious
Posts: 49
Joined: Sat Oct 13, 2012 6:38 am

Re: Why are projects 9415 and 9414 such low PPD?

Post by Luscious »

Something doesn't look right there. I have both linux and Win systems and the difference between actual memory and what is shown in my log is very small compared to 32 vs 23. Have you run a memory test or is your actual memory 24GB?
That missing 9.76GB is most likely my RAMDISK allocation. All memory sticks show up and CPU-Z shows 32GB installed.

I've only recently started running the CPU slot as it is usually paused anyway. The points difference is around +70K PPD with the CPU slot running and It's not impacting the GPU from what I can tell. MB slots are configured at x16/x8/x8/x8.

I'm on 361.43 - is it possible these projects are driver optimized? I'm scratching my head at what else to think.
toTOW
Site Moderator
Posts: 6296
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:

Re: Why are projects 9415 and 9414 such low PPD?

Post by toTOW »

Variability in PPD is expected ... and there is more variability on Windows because of the design of NV OpenCL that is more CPU and PCIe bandwidth dependant in Windows than it is in Linux.

I'm getting 380k PPD on my 980 (Windows 10), which is in the expected PPD range for this GPU (350-450k). Target is 420k. CPU feeding the GPU is a, i7 860 @ 3.5 GHz.

My 1070 is getting 600k PPD on these projects (Windows 10 too), which is in the expected range for this GPU (550-650k). Target is 620k. CPU feeding the GPU is a, i7 920 @ 3.5 GHz.

I don't know how the compare to such old drivers, but I noticed that 387.92 are faster than previous version of drivers that were continuously degrading performances update after update ... My systems are now all hitting their lowest average PPDs according to HFM.
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
Luscious
Posts: 49
Joined: Sat Oct 13, 2012 6:38 am

Re: Why are projects 9415 and 9414 such low PPD?

Post by Luscious »

Back with an update here and it's evident projects 11432 and 11713 are putting my cards back at their previous performance level.

No system changes made whatsoever.

Now I am curious if someone in the know can chime in to explain what the differences are between projects 9415/9414 versus 11432/11713, and why those differences are impacting scoring by 25-33% for me.
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Why are projects 9415 and 9414 such low PPD?

Post by foldy »

I see the same lower PPD for 94xx work units, 11xxx work units get more PPD.

Linux gets better PPD than Windows, I don't know if that is an option for you?
Luscious
Posts: 49
Joined: Sat Oct 13, 2012 6:38 am

Re: Why are projects 9415 and 9414 such low PPD?

Post by Luscious »

foldy wrote:Linux gets better PPD than Windows, I don't know if that is an option for you?
Unfortunately no, not while the cards are in the same box as my main rig, and dual booting CentOS is out of the question too because of how my SSD is sized and partitioned. Not sure if VM would be an option here.

Since I don't want to set up separate rigs or deal with water cooling a rack, I would pretty much have to look at a case that could fit two motherboards. Maybe in the future :biggrin:
Kuno
Posts: 31
Joined: Sat Sep 23, 2017 4:59 pm

Re: Why are projects 9415 and 9414 such low PPD?

Post by Kuno »

VM is not an option as the only VM that gives linux full access is hyper-visor in Server 2016, or if you have GRID compatible cards. If you had linux installed as a base, it wouldn't be a problem but going the other way, is literally impossible. Windows as a host OS and linux as a guest with full hardware access is impossible, with consumer grade hardware/software.
toTOW
Site Moderator
Posts: 6296
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:

Re: Why are projects 9415 and 9414 such low PPD?

Post by toTOW »

Luscious wrote:Now I am curious if someone in the know can chime in to explain what the differences are between projects 9415/9414 versus 11432/11713, and why those differences are impacting scoring by 25-33% for me.
First big difference is in the atom counts. Less atoms usually means poor scaling on higher end GPUs (I'm not afraid to call this a "CPU limited" situation).

I think there is another variable, but no one could ever show me clearly which one : simulation parameters are probably different (forcefield used, integrator, ...).
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Why are projects 9415 and 9414 such low PPD?

Post by bruce »

toTOW wrote:Less atoms usually means poor scaling on higher end GPUs ...
Actually, for those who run mid-range GPUs, these projects scale quite well. Thus it's reasonable to ask why projects OTHER THAN 9414 and 9415 scale higher than they should. It does make sense for FAH to hold the line against points inflation (thereby degrading the value of points earned earlier) so reducing the points on higher scaling projects does make sense. Is that what you want?
Post Reply