NVIDIA QUADRO 2000 suddenly stopped folding!

Driver issues associated with the Windows 10 roll-out

Moderators: Site Moderators, FAHC Science Team

Post Reply
neil_en_suite
Posts: 3
Joined: Sun May 17, 2020 6:19 pm

NVIDIA QUADRO 2000 suddenly stopped folding!

Post by neil_en_suite »

Hello,
first of all sorry if this post is similar to any other, I have been looking through the posts but not found a solution thats worked yet.

I have an old CAD station with a NVIDIA QUADRO 2000 (GF 106GL)
Up until a week ago, the card was folding well. I gave the machine periods of rest also to ensure I avoided any overheating.
I have had a windows update recently but the card continued to fold, then suddenly one day it stopped.

I have taken a snap of the log file, when the fold process starts up and you will see some error messages but I am not sure what they mean.
Unfortunately I dont have any only image account to link images to but needless to say on both the browser interface and the client interface the folding status looks as if paused.
- On the browser the indicator is yellow, with the status bar at 0%; work Unit(PRCG) = 0 ; work Unit(eta) = 0
- on the client under the GPU; The folding slot says Ready; The work ques says 'download'; all the info under the Selected work Unit pane, wither says 0 or unknown.

- I have re-installed the windows driver and it installed the same as before however the only difference I see is the driver says QUADRO 2000 with a "D" on the end
- I have downloaded and installed the NVIDIA driver mgmt software which I believe also updates the driver.
- I have uninstalled the FAH client, restarted then installed the latest FAH client
- I followed another forum post that suggested to delete the folder assigned tot he GPU within the FAH work folder within program files. Unfortunately there was no folder to delete.

None of these steps have worked.

I notice within the FAH work directory, there is only a folder for the CPU "01"; occasionally another folder appears and then disappears. this must be the client attempting to create a job for the GPU.

Not sure what else to do, so any suggestions or advice would be gratefully received.
Thanks in advance anyway for any help.


Below is the log file, from when the client first starts up. Near the bottom are entries with errors.

Code: Select all

  *********************** Log Started 2020-05-16T09:09:04Z ***********************
09:09:04:Trying to access database...
09:09:04:Successfully acquired database lock
09:09:04:Read GPUs.txt
09:09:05:Enabled folding slot 00: READY cpu:7
09:09:05:Enabled folding slot 01: READY gpu:0:GF106GL [Quadro 2000]
09:09:05:****************************** FAHClient ******************************
09:09:05:        Version: 7.6.13
09:09:05:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
09:09:05:      Copyright: 2020 foldingathome.org
09:09:05:       Homepage: https://foldingathome.org/
09:09:05:           Date: Apr 27 2020
09:09:05:           Time: 21:21:01
09:09:05:       Revision: 5a652817f46116b6e135503af97f18e094414e3b
09:09:05:         Branch: master
09:09:05:       Compiler: Visual C++ 2008
09:09:05:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
09:09:05:       Platform: win32 10
09:09:05:           Bits: 32
09:09:05:           Mode: Release
09:09:05:           Args: --open-web-control
09:09:05:         Config: C:\Users\neil_\AppData\Roaming\FAHClient\config.xml
09:09:05:******************************** CBang ********************************
09:09:05:           Date: Apr 24 2020
09:09:05:           Time: 17:07:55
09:09:05:       Revision: ea081a3b3b0f4a37c4d0440b4f1bc184197c7797
09:09:05:         Branch: master
09:09:05:       Compiler: Visual C++ 2008
09:09:05:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
09:09:05:       Platform: win32 10
09:09:05:           Bits: 32
09:09:05:           Mode: Release
09:09:05:******************************* System ********************************
09:09:05:            CPU: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz
09:09:05:         CPU ID: GenuineIntel Family 6 Model 45 Stepping 7
09:09:05:           CPUs: 8
09:09:05:         Memory: 19.92GiB
09:09:05:    Free Memory: 17.20GiB
09:09:05:        Threads: WINDOWS_THREADS
09:09:05:     OS Version: 6.2
09:09:05:    Has Battery: false
09:09:05:     On Battery: false
09:09:05:     UTC Offset: 1
09:09:05:            PID: 1420
09:09:05:            CWD: C:\Users\neil_\AppData\Roaming\FAHClient
09:09:05:  Win32 Service: false
09:09:05:             OS: Windows 10 Enterprise
09:09:05:        OS Arch: AMD64
09:09:05:           GPUs: 1
09:09:05:          GPU 0: Bus:4 Slot:0 Func:0 NVIDIA:1 GF106GL [Quadro 2000]
09:09:05:  CUDA Device 0: Platform:0 Device:0 Bus:4 Slot:0 Compute:2.1 Driver:8.0
09:09:05:OpenCL Device 0: Platform:0 Device:0 Bus:4 Slot:0 Compute:1.1 Driver:377.83
09:09:05:******************************* libFAH ********************************
09:09:05:           Date: Apr 15 2020
09:09:05:           Time: 14:53:14
09:09:05:       Revision: 216968bc7025029c841ed6e36e81a03a316890d3
09:09:05:         Branch: master
09:09:05:       Compiler: Visual C++ 2008
09:09:05:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
09:09:05:       Platform: win32 10
09:09:05:           Bits: 32
09:09:05:           Mode: Release
09:09:05:***********************************************************************
09:09:05:<config>
09:09:05:  <!-- Network -->
09:09:05:  <proxy v=':8080'/>
09:09:05:
09:09:05:  <!-- Slot Control -->
09:09:05:  <power v='full'/>
09:09:05:
09:09:05:  <!-- User Information -->
09:09:05:  <passkey v='*****'/>
09:09:05:  <team v='262280'/>
09:09:05:  <user v='neil_en_suite'/>
09:09:05:
09:09:05:  <!-- Folding Slots -->
09:09:05:  <slot id='0' type='CPU'/>
09:09:05:  <slot id='1' type='GPU'/>
09:09:05:</config>
09:09:05:WU01:FS00:Starting
09:09:05:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\neil_\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/v7/win/64bit/avx/Core_a7.fah/FahCore_a7.exe -dir 01 -suffix 01 -version 706 -lifeline 1420 -checkpoint 15 -np 7
09:09:05:WU01:FS00:Started FahCore on PID 13268
09:09:05:WU01:FS00:Core PID:3476
09:09:05:WU01:FS00:FahCore 0xa7 started
09:09:06:WU01:FS00:0xa7:*********************** Log Started 2020-05-16T09:09:05Z ***********************
09:09:06:WU01:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
09:09:06:WU01:FS00:0xa7:       Type: 0xa7
09:09:06:WU01:FS00:0xa7:       Core: Gromacs
09:09:06:WU01:FS00:0xa7:       Args: -dir 01 -suffix 01 -version 706 -lifeline 13268 -checkpoint 15 -np
09:09:06:WU01:FS00:0xa7:             7
09:09:06:WU01:FS00:0xa7:************************************ CBang *************************************
09:09:06:WU01:FS00:0xa7:       Date: Oct 26 2019
09:09:06:WU01:FS00:0xa7:       Time: 01:38:25
09:09:06:WU01:FS00:0xa7:   Revision: c46a1a011a24143739ac7218c5a435f66777f62f
09:09:06:WU01:FS00:0xa7:     Branch: master
09:09:06:WU01:FS00:0xa7:   Compiler: Visual C++ 2008
09:09:06:WU01:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
09:09:06:WU01:FS00:0xa7:   Platform: win32 10
09:09:06:WU01:FS00:0xa7:       Bits: 64
09:09:06:WU01:FS00:0xa7:       Mode: Release
09:09:06:WU01:FS00:0xa7:************************************ System ************************************
09:09:06:WU01:FS00:0xa7:        CPU: Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz
09:09:06:WU01:FS00:0xa7:     CPU ID: GenuineIntel Family 6 Model 45 Stepping 7
09:09:06:WU01:FS00:0xa7:       CPUs: 8
09:09:06:WU01:FS00:0xa7:     Memory: 19.92GiB
09:09:06:WU01:FS00:0xa7:Free Memory: 17.19GiB
09:09:06:WU00:FS01:Connecting to assign1.foldingathome.org:80
09:09:06:WU01:FS00:0xa7:    Threads: WINDOWS_THREADS
09:09:06:WU01:FS00:0xa7: OS Version: 6.2
09:09:06:WU01:FS00:0xa7:Has Battery: false
09:09:06:WU01:FS00:0xa7: On Battery: false
09:09:06:WU01:FS00:0xa7: UTC Offset: 1
09:09:06:WU01:FS00:0xa7:        PID: 3476
09:09:06:WU01:FS00:0xa7:        CWD: C:\Users\neil_\AppData\Roaming\FAHClient\work
09:09:06:WU01:FS00:0xa7:******************************** Build - libFAH ********************************
09:09:06:WU01:FS00:0xa7:    Version: 0.0.18
09:09:06:WU01:FS00:0xa7:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
09:09:06:WU01:FS00:0xa7:  Copyright: 2019 foldingathome.org
09:09:06:WU01:FS00:0xa7:   Homepage: https://foldingathome.org/
09:09:06:WU01:FS00:0xa7:       Date: Oct 26 2019
09:09:06:WU01:FS00:0xa7:       Time: 01:52:30
09:09:06:WU01:FS00:0xa7:   Revision: c1e3513b1bc0c16013668f2173ee969e5995b38e
09:09:06:WU01:FS00:0xa7:     Branch: master
09:09:06:WU01:FS00:0xa7:   Compiler: Visual C++ 2008
09:09:06:WU01:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
09:09:06:WU01:FS00:0xa7:   Platform: win32 10
09:09:06:WU01:FS00:0xa7:       Bits: 64
09:09:06:WU01:FS00:0xa7:       Mode: Release
09:09:06:WU01:FS00:0xa7:************************************ Build *************************************
09:09:06:WU01:FS00:0xa7:       SIMD: avx_256
09:09:06:WU01:FS00:0xa7:********************************************************************************
09:09:06:WU01:FS00:0xa7:Project: 16801 (Run 13, Clone 955, Gen 23)
09:09:06:WU01:FS00:0xa7:Unit: 0x0000001e82ed0b915e959440c5e34827
09:09:06:WU01:FS00:0xa7:Digital signatures verified
09:09:06:WU01:FS00:0xa7:Reducing thread count from 7 to 6 to avoid domain decomposition by a prime number > 3
09:09:06:WU01:FS00:0xa7:Calling: mdrun -s frame23.tpr -o frame23.trr -cpi state.cpt -cpt 15 -nt 6
09:09:06:WU01:FS00:0xa7:Steps: first=11500000 total=500000
09:09:07:WU01:FS00:0xa7:Completed 154517 out of 500000 steps (30%)
09:09:08:WU00:FS01:Assigned to work server 192.0.2.1
09:09:08:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GF106GL [Quadro 2000] from 192.0.2.1
09:09:08:WU00:FS01:Connecting to 192.0.2.1:8080
09:09:08:4:127.0.0.1:New Web session
09:09:29:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
09:09:29:WU00:FS01:Connecting to 192.0.2.1:80
09:09:42:WU01:FS00:0xa7:Completed 155000 out of 500000 steps (31%)
09:09:50:ERROR:WU00:FS01:Exception: Failed to connect to 192.0.2.1:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
09:09:50:WU00:FS01:Connecting to assign1.foldingathome.org:80
09:09:50:WU00:FS01:Assigned to work server 192.0.2.1
09:09:50:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GF106GL [Quadro 2000] from 192.0.2.1
09:09:50:WU00:FS01:Connecting to 192.0.2.1:8080
09:10:11:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
09:10:11:WU00:FS01:Connecting to 192.0.2.1:80
09:10:33:ERROR:WU00:FS01:Exception: Failed to connect to 192.0.2.1:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
09:10:50:WU00:FS01:Connecting to assign1.foldingathome.org:80
09:10:50:WU00:FS01:Assigned to work server 192.0.2.1
09:10:50:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GF106GL [Quadro 2000] from 192.0.2.1
09:10:50:WU00:FS01:Connecting to 192.0.2.1:8080
Joe_H
Site Admin
Posts: 7856
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: NVIDIA QUADRO 2000 suddenly stopped folding!

Post by Joe_H »

This response - viewtopic.php?f=74&t=35220&p=334019#p334019, and the topic it is in applies. I will bring your post to his attention, the Quadro 2000 may have been misclassified during the changes mentioned.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
neil_en_suite
Posts: 3
Joined: Sun May 17, 2020 6:19 pm

Re: NVIDIA QUADRO 2000 suddenly stopped folding!

Post by neil_en_suite »

Thank you. I will have a read.
FireFox-89
Posts: 72
Joined: Wed Nov 27, 2019 9:18 pm
Hardware configuration: Luna-
AMD Ryzen 5 2600 @ stock | Noctua NH-U9s with 2 Noctua B9 Redux 1600 92mm Fans
MSI B450 Tomahawk Max Revision 1.0 BIOS version 3.F0
Corsair Vengeance LPX 4x8GB DDR4-3000 | Timings 16-18-18-36
Sapphire Radeon RX 590 Nitro+ 8GB
SeaSonic M12II 850w 80+Bronze PSU

Terra-
Intel Core i5-4690K @ 4.20GHz with Vcore of 1.138v | Corsair H100i V2 with 2 Corsair SP120 120mm fans
Gigabyte Z87X-UD3H BIOS version 10b
Crucial Ballistix Sport 2x4GB DDR3-1600 | Timings 9-9-9-24
Sapphire Radeon RX 570 Pulse 4GB -100mV undervolt
OCZ Technology ZS Series 550w 80+Bronze PSU

Anubis-
Intel Core i7-5820K @3.80GHz with Vcore of 1.089v | Cooler Master Hyper 212 with Noctua NF-F12 120mm IPPC 3000 fan
Gigabyte X99-SOC Champion BIOS version F23c
G.Skill Trident Z RGB 2x8GB DDR4-3000 @ 2400 | Timings 15-15-15-36
Sapphire Radeon RX 580 Nitro+ 4GB -25mV undervolt
SeaSonic Prime Platinum 750w 80+Platinum PSU

All machines running F@H v7.6.21
Location: Lincolnshire, UK

Re: NVIDIA QUADRO 2000 suddenly stopped folding!

Post by FireFox-89 »

Also since it is an old card I would be tempted to strip the card down and clean it out along with some new thermal paste during its downtime.
Do you even fold bro?
Image
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: NVIDIA QUADRO 2000 suddenly stopped folding!

Post by bruce »

Fixed.
:oops:
Post Reply