GPU won't fold

Moderators: Site Moderators, FAHC Science Team

svanslyck
Posts: 6
Joined: Wed Sep 18, 2019 9:16 pm

GPU won't fold

Post by svanslyck »

Dunno where to post this; nothing else seemed appropriate.

Booted up this morning, started the client, and it's stuck on "Download." Dunno even what question to ask. Here's my log:

Code: Select all

21:14:15:Removing old file 'configs/config-20190918-110823.xml'
21:14:15:Saving configuration to config.xml
21:14:15:<config>
21:14:15:  <!-- Folding Core -->
21:14:15:  <checkpoint v='5'/>
21:14:15:
21:14:15:  <!-- Folding Slot Configuration -->
21:14:15:  <cause v='ALZHEIMERS'/>
21:14:15:  <opencl-index v='-1'/>
21:14:15:
21:14:15:  <!-- Logging -->
21:14:15:  <verbosity v='5'/>
21:14:15:
21:14:15:  <!-- Network -->
21:14:15:  <proxy v=':8080'/>
21:14:15:
21:14:15:  <!-- User Information -->
21:14:15:  <passkey v='********************************'/>
21:14:15:  <team v='223518'/>
21:14:15:  <user v='Steve_VanSlyck'/>
21:14:15:
21:14:15:  <!-- Folding Slots -->
21:14:15:  <slot id='1' type='GPU'/>
21:14:15:</config>
21:14:47:Removing old file 'configs/config-20190918-110854.xml'
21:14:47:Saving configuration to config.xml
21:14:47:<config>
21:14:47:  <!-- Folding Core -->
21:14:47:  <checkpoint v='5'/>
21:14:47:
21:14:47:  <!-- Folding Slot Configuration -->
21:14:47:  <cause v='ALZHEIMERS'/>
21:14:47:  <opencl-index v='-1'/>
21:14:47:
21:14:47:  <!-- Logging -->
21:14:47:  <verbosity v='5'/>
21:14:47:
21:14:47:  <!-- Network -->
21:14:47:  <proxy v=':8080'/>
21:14:47:
21:14:47:  <!-- User Information -->
21:14:47:  <passkey v='********************************'/>
21:14:47:  <team v='223518'/>
21:14:47:  <user v='Steve_VanSlyck'/>
21:14:47:
21:14:47:  <!-- Folding Slots -->
21:14:47:  <slot id='1' type='GPU'/>
21:14:47:</config>
21:14:56:WU01:FS01:Starting
21:14:56:ERROR:WU01:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
21:19:10:WU01:FS01:Starting
21:19:10:ERROR:WU01:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
21:19:10:WU01:FS01:Sending unit results: id:01 state:SEND error:FAILED project:14250 run:41 clone:1 gen:5 core:0x21 unit:0x0000000680fccb0a5d6ed217db6968b8
21:19:11:WU01:FS01:Connecting to 128.252.203.10:8080
21:19:11:WU01:FS01:Server responded WORK_ACK (400)
21:19:11:WU01:FS01:Cleaning up
21:19:16:WU00:FS01:Connecting to 65.254.110.245:8080
21:19:17:WU00:FS01:Assigned to work server 155.247.166.220
21:19:17:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:TU102 [GeForce RTX 2080 Ti Rev. A] M 13448 from 155.247.166.220
21:19:17:WU00:FS01:Connecting to 155.247.166.220:8080
21:19:18:WU00:FS01:Downloading 3.18MiB
21:19:19:WU00:FS01:Download complete
21:19:19:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:14186 run:2 clone:1 gen:270 core:0x21 unit:0x000001630002894c5d389735718426aa
21:19:19:WU00:FS01:Starting
21:19:19:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
21:19:19:WU00:FS01:Starting
21:19:19:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
Mod edit: added Code tags to logfile, post moved to appropriate forum
Joe_H
Site Admin
Posts: 7856
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: GPU won't fold

Post by Joe_H »

Welcome to the folding support forum.

You have provided us some of the information needed to help you solve this problem, please add the following information:

The beginning 2-300 lines of the log file that show the hardware, system, and folding configuration information. Before doing so, please return the logging verbosity back to the default value of 3.

Have you made any changes to your system recently such as hardware changes or software updates.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: GPU won't fold

Post by bruce »

Early versions of FAH have had difficulties recovering from a temporary communications outage ... even ones that were very brief .. if they happened when FAHClient was actually uploading/download something (which a relatively rare percentage of the time. I don't know if all of those bugs have been quashed.

The simplest way to deal with them is to reboot unless you can find and fix something that's interrupting your communications (like a loosed internet connection somewhere or a router that decides to reboot).
toTOW
Site Moderator
Posts: 6296
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:

Re: GPU won't fold

Post by toTOW »

It's not a download issue : the core fails to start because it doesn't find the GPU (or an OpenCL driver).

Did you get a Windows update when you rebooted ? Try to reinstall your NV drivers from NV website.
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
svanslyck
Posts: 6
Joined: Wed Sep 18, 2019 9:16 pm

Re: GPU won't fold

Post by svanslyck »

No system changes. Multiple reboots in attempt to fix.

Using Linux. Didn't update anything.

Everything worked fine until I booted up one day and then this occurred. Today's log file:

Code: Select all

*********************** Log Started 2019-09-21T22:13:21Z ***********************
22:13:21:************************* Folding@home Client *************************
22:13:21:      Website: https://foldingathome.org/
22:13:21:    Copyright: (c) 2009-2018 foldingathome.org
22:13:21:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
22:13:21:         Args: 
22:13:21:       Config: /home/raziel/config.xml
22:13:21:******************************** Build ********************************
22:13:21:      Version: 7.5.1
22:13:21:         Date: May 12 2018
22:13:21:         Time: 22:51:07
22:13:21:   Repository: Git
22:13:21:     Revision: 4705bf53c635f88b8fe85af7675557e15d491ff0
22:13:21:       Branch: master
22:13:21:     Compiler: GNU 4.4.7 20120313 (Red Hat 4.4.7-18)
22:13:21:      Options: -std=gnu++98 -O3 -funroll-loops
22:13:21:     Platform: linux2 4.14.0-3-amd64
22:13:21:         Bits: 64
22:13:21:         Mode: Release
22:13:21:******************************* System ********************************
22:13:21:          CPU: AMD Ryzen 9 3900X 12-Core Processor
22:13:21:       CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
22:13:21:         CPUs: 24
22:13:21:       Memory: 62.89GiB
22:13:21:  Free Memory: 59.89GiB
22:13:21:      Threads: POSIX_THREADS
22:13:21:   OS Version: 5.2
22:13:21:  Has Battery: false
22:13:21:   On Battery: false
22:13:21:   UTC Offset: -4
22:13:21:          PID: 10424
22:13:21:          CWD: /home/raziel
22:13:21:           OS: Linux 5.2.8-200.fc30.x86_64 x86_64
22:13:21:      OS Arch: AMD64
22:13:21:         GPUs: 1
22:13:21:        GPU 0: Bus:11 Slot:0 Func:0 NVIDIA:7 TU102 [GeForce RTX 2080 Ti Rev. A]
22:13:21:               M 13448
22:13:21:CUDA Device 0: Platform:0 Device:0 Bus:11 Slot:0 Compute:7.5 Driver:10.1
22:13:21:       OpenCL: Not detected: Failed to open dynamic library 'libOpenCL.so':
22:13:21:               libOpenCL.so: cannot open shared object file: No such file or
22:13:21:               directory
22:13:21:***********************************************************************
22:13:21:<config>
22:13:21:  <!-- Folding Core -->
22:13:21:  <checkpoint v='5'/>
22:13:21:
22:13:21:  <!-- Folding Slot Configuration -->
22:13:21:  <cause v='ALZHEIMERS'/>
22:13:21:  <opencl-index v='-1'/>
22:13:21:
22:13:21:  <!-- Network -->
22:13:21:  <proxy v=':8080'/>
22:13:21:
22:13:21:  <!-- User Information -->
22:13:21:  <passkey v='********************************'/>
22:13:21:  <team v='223518'/>
22:13:21:  <user v='Steve_VanSlyck'/>
22:13:21:
22:13:21:  <!-- Folding Slots -->
22:13:21:  <slot id='1' type='GPU'/>
22:13:21:</config>
22:13:21:Trying to access database...
22:13:51:ERROR:Exception: Error executing: 'PRAGMA synchronous=NORMAL': database is locked
Mod Edit: Added Code Tags - PantherX
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: GPU won't fold

Post by bruce »

22:13:21: OpenCL: Not detected: Failed to open dynamic library 'libOpenCL.so':
22:13:21: libOpenCL.so: cannot open shared object file: No such file or directory
Go to https://www.khronos.org/ and install the opencl developer package. (There are other methods that work, too.)
svanslyck
Posts: 6
Joined: Wed Sep 18, 2019 9:16 pm

Re: GPU won't fold

Post by svanslyck »

Any idea why I hadn't needed openCL until now?
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: GPU won't fold

Post by bruce »

OpenCL has always been needed by the FAHCore for GPUs. The question really is why you had it previously.

The runtime code for OpenCL is often packaged with the drivers for the GPU -- or it might have been packaged with a distro you were running previously.

The run-time code is all that's really needed (not the full developer package) but if you do install the developer package, you'll certainly get the run-time code installed properly.

On a Windows system (not your Linux system) the same issue comes up and the runtime driver is included if you get your drivers from NVidia. On my Desktop Linux distro, I've had troubles installing drivers directly fron NVidia (they do work on the server distro) so I don't recommend that option. Instead, I choose a pre-packaged .deb containing the proprietary drivers. The availability of that option depends on which distro you're running and you didn't include the first 200 lines of FAH's log where that information is shown.

Too Much Information? ? ?
svanslyck
Posts: 6
Joined: Wed Sep 18, 2019 9:16 pm

Re: GPU won't fold

Post by svanslyck »

Well I reinstalled both nvidia and cuda, which took forever and didn't help, whether opencl-index was set to 1 or to 0. Isn't there some way to just unlock the database? That appears to be the error.

Code: Select all

23:55:35:Removing old file 'configs/config-20190918-210555.xml'
23:55:35:Saving configuration to config.xml
23:55:35:<config>
23:55:35:  <!-- Folding Core -->
23:55:35:  <checkpoint v='5'/>
23:55:35:
23:55:35:  <!-- Folding Slot Configuration -->
23:55:35:  <cause v='ALZHEIMERS'/>
23:55:35:  <opencl-index v='0'/>
23:55:35:
23:55:35:  <!-- Network -->
23:55:35:  <proxy v=':8080'/>
23:55:35:
23:55:35:  <!-- User Information -->
23:55:35:  <passkey v='********************************'/>
23:55:35:  <team v='223518'/>
23:55:35:  <user v='Steve_VanSlyck'/>
23:55:35:
23:55:35:  <!-- Folding Slots -->
23:55:35:  <slot id='1' type='GPU'/>
23:55:35:</config>
23:55:46:Removing old file 'configs/config-20190918-210612.xml'
23:55:46:Saving configuration to config.xml
23:55:46:<config>
23:55:46:  <!-- Folding Core -->
23:55:46:  <checkpoint v='5'/>
23:55:46:
23:55:46:  <!-- Folding Slot Configuration -->
23:55:46:  <cause v='ALZHEIMERS'/>
23:55:46:  <opencl-index v='0'/>
23:55:46:
23:55:46:  <!-- Network -->
23:55:46:  <proxy v=':8080'/>
23:55:46:
23:55:46:  <!-- User Information -->
23:55:46:  <passkey v='********************************'/>
23:55:46:  <team v='223518'/>
23:55:46:  <user v='Steve_VanSlyck'/>
23:55:46:
23:55:46:  <!-- Folding Slots -->
23:55:46:  <slot id='1' type='GPU'/>
23:55:46:</config>
23:56:21:WU00:FS01:Starting
23:56:21:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
23:56:26:25:127.0.0.1:New Web connection
Mod Edit: Added Code Tags - PantherX
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: GPU won't fold

Post by bruce »

Please post the FIRST 200 lines of the log. You're not starting from the beginning of the log.

In FAHControl, go to the log tab. Uncheck Follow and click Refresh. Copy the log beginning from ***** System *****
svanslyck
Posts: 6
Joined: Wed Sep 18, 2019 9:16 pm

Re: GPU won't fold

Post by svanslyck »

I don't know why it isn't copying the whole log.

Code: Select all

*********************** Log Started 2019-09-24T10:46:47Z ***********************
10:46:47:************************* Folding@home Client *************************
10:46:47:      Website: https://foldingathome.org/
10:46:47:    Copyright: (c) 2009-2018 foldingathome.org
10:46:47:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
10:46:47:         Args: 
10:46:47:       Config: /home/raziel/config.xml
10:46:47:******************************** Build ********************************
10:46:47:      Version: 7.5.1
10:46:47:         Date: May 12 2018
10:46:47:         Time: 22:51:07
10:46:47:   Repository: Git
10:46:47:     Revision: 4705bf53c635f88b8fe85af7675557e15d491ff0
10:46:47:       Branch: master
10:46:47:     Compiler: GNU 4.4.7 20120313 (Red Hat 4.4.7-18)
10:46:47:      Options: -std=gnu++98 -O3 -funroll-loops
10:46:47:     Platform: linux2 4.14.0-3-amd64
10:46:47:         Bits: 64
10:46:47:         Mode: Release
10:46:47:******************************* System ********************************
10:46:47:          CPU: AMD Ryzen 9 3900X 12-Core Processor
10:46:47:       CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
10:46:47:         CPUs: 24
10:46:47:       Memory: 62.89GiB
10:46:47:  Free Memory: 60.05GiB
10:46:47:      Threads: POSIX_THREADS
10:46:47:   OS Version: 5.2
10:46:47:  Has Battery: false
10:46:47:   On Battery: false
10:46:47:   UTC Offset: -4
10:46:47:          PID: 3656
10:46:47:          CWD: /home/raziel
10:46:47:           OS: Linux 5.2.8-200.fc30.x86_64 x86_64
10:46:47:      OS Arch: AMD64
10:46:47:         GPUs: 1
10:46:47:        GPU 0: Bus:11 Slot:0 Func:0 NVIDIA:7 TU102 [GeForce RTX 2080 Ti Rev. A]
10:46:47:               M 13448
10:46:47:CUDA Device 0: Platform:0 Device:0 Bus:11 Slot:0 Compute:7.5 Driver:10.1
10:46:47:       OpenCL: Not detected: clGetDeviceIDs() returned -1
10:46:47:***********************************************************************
10:46:47:<config>
10:46:47:  <!-- Folding Core -->
10:46:47:  <checkpoint v='5'/>
10:46:47:
10:46:47:  <!-- Folding Slot Configuration -->
10:46:47:  <cause v='ALZHEIMERS'/>
10:46:47:  <opencl-index v='0'/>
10:46:47:
10:46:47:  <!-- Network -->
10:46:47:  <proxy v=':8080'/>
10:46:47:
10:46:47:  <!-- User Information -->
10:46:47:  <passkey v='********************************'/>
10:46:47:  <team v='223518'/>
10:46:47:  <user v='Steve_VanSlyck'/>
10:46:47:
10:46:47:  <!-- Folding Slots -->
10:46:47:  <slot id='1' type='GPU'/>
10:46:47:</config>
10:46:47:Trying to access database...
10:46:47:Successfully acquired database lock
10:46:47:Enabled folding slot 01: READY gpu:0:TU102 [GeForce RTX 2080 Ti Rev. A] M 13448
10:46:47:WU00:FS01:Starting
10:46:47:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
10:46:47:WU00:FS01:Starting
10:46:47:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
10:47:47:WU00:FS01:Starting
10:47:47:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
10:47:56:27:127.0.0.1:New Web connection
10:49:24:WU00:FS01:Starting
10:49:24:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
10:52:01:WU00:FS01:Starting
10:52:01:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually
Mod Edit: Added Code Tags - PantherX
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: GPU won't fold

Post by bruce »

10:46:47: GPUs: 1
10:46:47: GPU 0: Bus:11 Slot:0 Func:0 NVIDIA:7 TU102 [GeForce RTX 2080 Ti Rev. A]
10:46:47: M 13448
10:46:47:CUDA Device 0: Platform:0 Device:0 Bus:11 Slot:0 Compute:7.5 Driver:10.1
10:46:47: OpenCL: Not detected: clGetDeviceIDs() returned

You need to install an OpenCL runtime package. (The CUDA drivers are not particularly useful for FAH unless you also get OpenCL)

I'm not sure why that is so difficult to find, but it is. I usually end up recommending the OpenCL developer's package from Khronos.org although it includes a lot more stuff than just the runtime drivers.
DocJonz
Posts: 242
Joined: Thu Dec 06, 2007 6:31 pm
Hardware configuration: Folding with: 4x RTX 4070Ti, 1x RTX 3070
Location: United Kingdom
Contact:

Re: GPU won't fold

Post by DocJonz »

I had a similar issue a while back while using Ubuntu 18.04. I used the the following to fix the OpenCL issue;
sudo apt-get install olc-icd-opencl-dev

You can then check it has installed correctly by running;
sudo apt-get install clinfo
clinfo
Folding Stats (HFM.NET): DocJonz Folding Farm Stats
svanslyck
Posts: 6
Joined: Wed Sep 18, 2019 9:16 pm

Re: GPU won't fold

Post by svanslyck »

In any event I am totally lost here. I cannot find where to download it (lots of discussion on khronos.org (or whatever the correct spelling is, but no download links), the package is unknown to yum, and I'm giving up. If the software is needed I would've expected it to have been installed as a dependency by FAH or CUDA. Not sure why I should have to install something I didn't need the first two times I installed FAH on Linux, including this box.
DocJonz
Posts: 242
Joined: Thu Dec 06, 2007 6:31 pm
Hardware configuration: Folding with: 4x RTX 4070Ti, 1x RTX 3070
Location: United Kingdom
Contact:

Re: GPU won't fold

Post by DocJonz »

It is probably an issue with a graphics driver update - I have had occasions where the requied parts of the OpenCL package have not been present with the GPU driver.
Open a Terminal and type in the commands I gave on the previous post - see if that fixes it for you.
Folding Stats (HFM.NET): DocJonz Folding Farm Stats
Post Reply