Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

It seems that a lot of GPU problems revolve around specific versions of drivers. Though NVidia has their own support structure, you can often learn from information reported by others who fold.

Moderators: Site Moderators, FAHC Science Team

ilxli
Posts: 3
Joined: Tue May 08, 2018 10:43 am

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by ilxli »

Hey,

for me its still fine, I'm using Redshift, but soon it wont be, ill open a ticket!

: )
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by bruce »

I can't speak to what you might need for rendering. We're a FAH support forum, and this discussion is rapidly going off-topic except if it's somehow related to FAH.

According to the best information we have, There was a problem is the nVidia 397.31 driver when running FAH (and BOINC, for that matter). OpenCL and CUDA computing works for a while and then dies. It was enough of an issue that nVidia released a hotfix version 397.55 which may work and 397.64 has also been released. Be sure to do a clean install.

If your 730 worked before, it should still work, but the 1060 is going to run circles around it.
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by foldy »

bruce, this is a very special issue for people using mainboard with plx pcie switch chip PEX8747. There FAH crashes with newer nvidia drivers and last good version is nvidia 382.53
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by foldy »

There is now a similar open issue in nvidia 397.93
Random DPC watchdog violation errors occur when using multiple GPUs on motherboards with PLX chips. [2079538]
adnplayfilms
Posts: 1
Joined: Thu Jul 05, 2018 2:25 pm

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by adnplayfilms »

NEWS UPDATE

We are many professionals who, since last year, we are finding this problem.

In case there is anyone with any doubt we can confirm that the problem is the motherboards with PLX processor exclusively in a multi-GPU system.

The good news:

nVidia knows the problem and is working on it since version 397.93.

Code: Select all

http://us.download.nvidia.com/Windows/397.93/397.93-win10-win8-win7-desktop-release-notes.pdf
-----
"Open Issues in Version 397.93 WHQL
This section includes lists of issues
that are either not fixed or not implemented in this version. Some problems listed may
not have been thoroughly researched and, in fact, may not be NVIDIA issues. Others
may have workaround solutions.
► Random DPC watchdog violation errors occur when using multiple GPUs on
motherboards with PLX chips. [2079538] "
-----

The bad news:

The problem was not solved in version 398.11.

Code: Select all

http://us.download.nvidia.com/Windows/398.11/398.11-win10-win8-win7-desktop-release-notes.pdf
-----
"Open Issues in Version 398.11 WHQL
This section includes lists of issues
that are either not fixed or not implemented in this version. Some problems listed may
not have been thoroughly researched and, in fact, may not be NVIDIA issues. Others
may have workaround solutions.
► Random DPC watchdog violation errors occur when using multiple GPUs on
motherboards with PLX chips. [2079538] "
-----

And it still has no solution in the current version 398.36.

Code: Select all

http://us.download.nvidia.com/Windows/398.36/398.36-win10-win8-win7-desktop-release-notes.pdf
-----
"Open Issues in Version 398.36 WHQL
This section includes lists of issues
that are either not fixed or not implemented in this version. Some problems listed may
not have been thoroughly researched and, in fact, may not be NVIDIA issues. Others
may have workaround solutions.
► Random DPC watchdog violation errors occur when using multiple GPUs on
motherboards with PLX chips. [2079538] "
-----

I recommend that those affected, before updating drivers, check the Release Notes to see if they have solved the problem.

Code: Select all

http://us.download.nvidia.com/Windows/***.**/***.**-win10-win8-win7-desktop-release-notes.pdf
Change the asterisks by the version number

Greetings.
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by foldy »

Still not fixed but nvidia driver 398.82 mentions a possible workaround:
https://forums.geforce.com/default/topi ... d-8-1-18-/
-Random DPC watchdog violation error when using multiple GPUs on motherboards with PLX chips -> Use the following tool to enable MSI mode as a temporary workaround (only apply changes to each NVIDIA GPU installed):
https://github.com/CHEF-KOCH/MSI-utility/releases
MSI means "Message Signaled-Based Interrupts" here, not the company.
This tool edits Windows registry to change interrupt handling from default to MSI.

Create a Windows Restore Point before using this tool. So if anything goes wrong you can restore the previous registry settings.
Jimboc
Posts: 68
Joined: Sun Feb 12, 2012 11:43 am
Hardware configuration: Corsair Obsidian 750D Windows Airflow Edition

Intel Core i9 Extreme 7980XE @ 2.6 GHz (18 cores, 36 threads)

64 GB Corsair Dominator Platinum DDR4 RAM

2x Nvidia Titan RTX (NVLink Enabled) (Nvidia 526.98 Studio Driver)

Asus Rampage VI APEX (BIOS 1401)(Intel X299 Chipset)

Corsair AX1600i Titanium Plus Power Supply

Corsair Neutron NX500 800GB SSD (System Drive)

Seagate SkyHawk 10 TB , 256 MB Cache (Data Drive)

Creative Sound BlasterX AE-5 Plus Pure Edition

Dell UP3218K and Dell U2711

Windows 11 Pro for Workstations 64 Bit (Version 22H2)
Location: Ireland

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by Jimboc »

adnplayfilms wrote: I recommend that those affected, before updating drivers, check the Release Notes to see if they have solved the problem.

Code: Select all

http://us.download.nvidia.com/Windows/***.**/***.**-win10-win8-win7-desktop-release-notes.pdf
Change the asterisks by the version number

Greetings.
foldy wrote:Still not fixed but nvidia driver 398.82 mentions a possible workaround:
https://forums.geforce.com/default/topi ... d-8-1-18-/
-Random DPC watchdog violation error when using multiple GPUs on motherboards with PLX chips -> Use the following tool to enable MSI mode as a temporary workaround (only apply changes to each NVIDIA GPU installed):
https://github.com/CHEF-KOCH/MSI-utility/releases
MSI means "Message Signaled-Based Interrupts" here, not the company.
This tool edits Windows registry to change interrupt handling from default to MSI.

Create a Windows Restore Point before using this tool. So if anything goes wrong you can restore the previous registry settings.
Hi adnplayfilms and foldy,

Many thanks for keeping us posted on this issue. It's good to know that Nvidia are aware of it and have provided a workaround. Out of curiosity do you know if anyone suffering from this issue has resolved it using the tool you kindly linked to?

As for me; I'm happy to report my new Asus Rampage VI Apex (X299 chipset)(BIOS Version 1401) when used with my Intel Core i9 7980XE CPU, 64 GB of RAM and 2x Nvidia Geforce GTX Titans (from 2013 and 2014) work perfectly. I have not and I hope I won't experience this issue.

Edit/Update: 9th June 2019: I have now added a 64 GB Intel Optane Memory Accelerator module and replaced my Geforce GTX Titans with 2x Nvidia Titan RTX "T-Rex" with NVLink Enabled. The system continues to function perfectly. Folding performance is amazing! :D

Thanks again.
Last edited by Jimboc on Sun Jun 09, 2019 4:20 pm, edited 1 time in total.
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by foldy »

No, I don't know if the tool helped anyone for folding. Your mainboard does not have this issue as it has no plx pcie switch chip PEX8747. Your mainboard supports 4 GPUs at pcie 3.0 x8 because your CPU provides 44 pcie Lanes. The plx pcie switch chip PEX8747 is only available on some rare mainboards to support 4 GPUs at pcie 3.0 x8 running a cheap CPU with 16 lanes only.
Jimboc
Posts: 68
Joined: Sun Feb 12, 2012 11:43 am
Hardware configuration: Corsair Obsidian 750D Windows Airflow Edition

Intel Core i9 Extreme 7980XE @ 2.6 GHz (18 cores, 36 threads)

64 GB Corsair Dominator Platinum DDR4 RAM

2x Nvidia Titan RTX (NVLink Enabled) (Nvidia 526.98 Studio Driver)

Asus Rampage VI APEX (BIOS 1401)(Intel X299 Chipset)

Corsair AX1600i Titanium Plus Power Supply

Corsair Neutron NX500 800GB SSD (System Drive)

Seagate SkyHawk 10 TB , 256 MB Cache (Data Drive)

Creative Sound BlasterX AE-5 Plus Pure Edition

Dell UP3218K and Dell U2711

Windows 11 Pro for Workstations 64 Bit (Version 22H2)
Location: Ireland

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by Jimboc »

foldy wrote:No, I don't know if the tool helped anyone for folding. Your mainboard does not have this issue as it has no plx pcie switch chip PEX8747. Your mainboard supports 4 GPUs at pcie 3.0 x8 because your CPU provides 44 pcie Lanes. The plx pcie switch chip PEX8747 is only available on some rare mainboards to support 4 GPUs at pcie 3.0 x8 running a cheap CPU with 16 lanes only.
Hi foldy,

It's really impressive that you have been able narrow down the cause to that exact model of switch chip. Thanks for this and for clarifying which motherboards are/are not affected.
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by foldy »

Nvidia driver 411.63 wrongly says fixed issue: [GeForce GTX 1080Ti]: Random DPC watchdog violation error when using multiple GPUs on motherboards with PLX chips. [2079538] But it is not they are still working on it.

Workaround is using older nvidia driver 382.53 or setting "message signaled interrupt" in Windows registry but I don't know if this works for FAH.
https://forums.geforce.com/default/topi ... ues-help-/
ilxli
Posts: 3
Joined: Tue May 08, 2018 10:43 am

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by ilxli »

Hi there,
I finally have a solution with the latest NVIDIA drivers and quad GPU on the ASUS X299ws

Now running smoothly on ASUS X299ws with 2 x GeForce GTX 1080Ti and 2 x GeForce Titan X

Instaled 417.35-desktop-win10-64bit
Used MSI_util_v2 program to fix the problem.
You can find the program here: https://github.com/CHEF-KOCH/MSI-utility/releases

Hope this will help you all : )
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by foldy »

Thank you for reporting back.
Jimboc
Posts: 68
Joined: Sun Feb 12, 2012 11:43 am
Hardware configuration: Corsair Obsidian 750D Windows Airflow Edition

Intel Core i9 Extreme 7980XE @ 2.6 GHz (18 cores, 36 threads)

64 GB Corsair Dominator Platinum DDR4 RAM

2x Nvidia Titan RTX (NVLink Enabled) (Nvidia 526.98 Studio Driver)

Asus Rampage VI APEX (BIOS 1401)(Intel X299 Chipset)

Corsair AX1600i Titanium Plus Power Supply

Corsair Neutron NX500 800GB SSD (System Drive)

Seagate SkyHawk 10 TB , 256 MB Cache (Data Drive)

Creative Sound BlasterX AE-5 Plus Pure Edition

Dell UP3218K and Dell U2711

Windows 11 Pro for Workstations 64 Bit (Version 22H2)
Location: Ireland

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by Jimboc »

foldy wrote:No, I don't know if the tool helped anyone for folding. Your mainboard does not have this issue as it has no plx pcie switch chip PEX8747. Your mainboard supports 4 GPUs at pcie 3.0 x8 because your CPU provides 44 pcie Lanes. The plx pcie switch chip PEX8747 is only available on some rare mainboards to support 4 GPUs at pcie 3.0 x8 running a cheap CPU with 16 lanes only.
Agreed; I think you are spot on with this information. The 44 lanes of my motherboard continue to work flawlessly even with my new dual Titan RTX set up (I've updated my post above to state this board is still perfectly stable).
foldy wrote:Thank you for reporting back.
It's really good that this annoying was finally resolved. Thanks for reporting back.
ari2asem
Posts: 21
Joined: Sat Jun 08, 2019 1:07 am

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by ari2asem »

a question about MSI-utility....should I always run this small tool when I am folding? in the background running?
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Quad 1080Ti + ASUS X299 + F@H > DPC_WATCHDOG_VIOLATION

Post by foldy »

No the chefkoch tool changes some registry keys and you need to reboot - that's it
Post Reply