HOWTO: How I got my R9 290 folding on Linux

It seems that a lot of GPU problems revolve around specific versions of drivers. Though AMD has their own support structure, you can often learn from information reported by others who fold.

Moderators: Site Moderators, FAHC Science Team

HOWTO: How I got my R9 290 folding on Linux

Postby Frogging101 » Wed Mar 25, 2020 7:47 pm

My setup is as follows:

Code: Select all
Radeon R9 290 (Hawaii) with free software drivers (amdgpu/mesa)
Intel Core i7-4790
Linux Mint 18.1 64-bit
Linux kernel v5.5.1


I created this guide to share my findings and experience in setting up Folding@Home on this computer. I hope that it can provide some insight and help to others.

Installing the AMDGPU-PRO OpenCL implementation

I use the free software GPU driver stack. Unfortunately, I could not get Folding@Home to work with the free software OpenCL implementations from ROCm or Mesa. So I will not bother explaining how to set those up. If you have them, you may want to remove them in case they interfere.

What did work was something suggested by dschuermann on IRC (#radeon on freenode; great people there). This allows you to use the OpenCL implementation from the proprietary AMDGPU-PRO driver, without installing the whole driver.

  1. Get the AMDGPU-PRO driver package:
    1. Go to https://www.amd.com/en/support and select your graphics card.
    2. Scroll down to Ubuntu x86 64-bit and download the file. (currently: amdgpu-pro-19.50-967956-ubuntu-18.04.tar.xz)
    3. Extract it to an empty directory somewhere. Do not run the install script or anything else.
  2. Extract the OpenCL driver from the appropriate .deb file.
    For Vega 10 and later products, this is the .deb file matching the pattern opencl-amdgpu-pro-icd_*_amd64.deb.
    For products older than Vega 10, this is the .deb file matching the pattern opencl-orca-amdgpu-pro-icd_*_amd64.deb.
    1. In a terminal in the directory with the .deb files, run the following:
      Code: Select all
      # Extract the contents of the .deb package to the current directory
      dpkg -x DEB_FILE . # replace DEB_FILE with the appropriate pattern from above

      # Replace the path in the .icd file with the full path to where we're going to put the library
      sed -i 's|^|/opt/amdgpu-pro/lib/x86_64-linux-gnu/|' etc/OpenCL/vendors/amdocl*.icd

      # Copy the etc and opt directories extracted from the package to the system root.
      sudo cp -r etc opt /


Fixing the permissions issue (clGetDeviceIDs() returned -1)

Add the fahclient user to the "video" and "render" (if you have it) groups:
Code: Select all
sudo usermod -aG video fahclient
sudo usermod -aG render fahclient # don't worry if you don't have a "render" group


This is necessary because FAHClient must be able to access devices in /dev/dri, such as /dev/dri/card0 and /dev/dri/renderD128. Some distributions place the latter under the "render" group instead of the "video" group, so you may need both if your system is like that.

Some donors have been getting "OpenCL: Not detected: clGetDeviceIDs() returned -1" unless they run FAHClient as root, and adding the fahclient user to the video group doesn't fix it.

I observed this myself, and upon investigating, I discovered the problem. The official Folding@Home packages (at least the Debian-based ones) set up FAHClient to be launched by an init.d script, which launches FAHClient with "--run-as fahclient", so that it runs as the "fahclient" user.

The problem is that when FAHClient is launched with the --run-as option, it does not give itself the supplementary groups of the user passed into the --run-as option. Thus the FAHClient process will not have the permissions of the groups that the "fahclient" user is a member of. You can see this for yourself:
Code: Select all
$ pidof FAHClient
26801 26796
$ sudo cat /proc/26796/status | grep 'Groups:'
Groups:


The lack of any group numbers after "Groups:" shows that the FAHClient process does not have any supplementary groups, even though it is running as the "fahclient" user. This is due to a bug in FAHClient.

For the developers: If I had to guess (I have not verified this), this is probably happening because FAHClient uses setuid() to drop privileges with the "--run-as" argument. This is not sufficient. You must call initgroups() before setuid() to give the process the supplementary groups of the target user.

TL;DR For the users: To work around this, I created a systemd unit to start FAHClient instead of the init.d script. This works by using the systemd unit's "User" option to change users instead of having FAHClient do it.

First, stop FAHClient ("sudo systemctl stop FAHClient").

Create /etc/systemd/system/FAHClient.service (The name matters; calling it this will automatically supersede the init.d script of the same name), with the following contents:
Code: Select all
[Unit]
Description=Folding@Home Client
After=network.target

[Service]
Type=simple
ExecStart=/usr/bin/FAHClient /etc/fahclient/config.xml --pid-file=/var/run/fahclient/fahclient.pid --daemon
PIDFile=/var/run/fahclient/fahclient.pid
User=fahclient
RuntimeDirectory=fahclient
WorkingDirectory=~

[Install]
WantedBy=multi-user.target


Then do the following:
Code: Select all
sudo systemctl daemon-reload
sudo systemctl start FAHClient.service
# And, if you want it to start automatically on boot...
sudo systemctl enable FAHClient.service
Last edited by Frogging101 on Fri Oct 09, 2020 7:18 pm, edited 12 times in total.
Frogging101
 
Posts: 69
Joined: Wed Mar 25, 2020 3:39 am
Location: Canada

Re: How I got my R9 290 folding on Linux

Postby bruce » Wed Mar 25, 2020 8:24 pm

Thank you for the detailed HOWTO.

I suggest you make the same report at https://github.com/FoldingAtHome/fah-issues/issues. Someday the Developers will have time to examine those issues and fix most of them ... including things like your "This is a bug in FAH" mentioned above.

In the meantime, others will find it here.
bruce
 
Posts: 20126
Joined: Thu Nov 29, 2007 11:13 pm
Location: So. Cal.

Re: HOWTO: How I got my R9 290 folding on Linux

Postby Paragon » Sat Mar 28, 2020 1:11 am

Nice fix, I'll try it out next time I'm tinkering with my Ubuntu box (currently folding under root). I'll try to keep an eye out for people asking about this and point them here
Paragon
 
Posts: 138
Joined: Fri Oct 21, 2011 4:24 am
Location: United States

Re: HOWTO: How I got my R9 290 folding on Linux

Postby Revenant » Sat Mar 28, 2020 5:39 am

Very useful. Thank you for the post. Trying this out now.
ImageImage
User avatar
Revenant
 
Posts: 3
Joined: Thu Jun 09, 2011 12:37 am

Re: HOWTO: How I got my R9 290 folding on Linux

Postby dmk1000 » Sat Mar 28, 2020 6:09 pm

Thanks. This also worked for the Ryzen Vega 8 Mobile in my Ryzen 5 2400G.
I grabbed the AMD drives for Vega 56 an they've worked fine.
Image
dmk1000
 
Posts: 5
Joined: Sat Mar 28, 2020 6:03 pm

Re: HOWTO: How I got my R9 290 folding on Linux

Postby Rainmaker » Sat Mar 28, 2020 7:02 pm

I signed up just to say thank you for this! I primarily run Linux on my desktop/workstation (ThreadRipper 3960X, Vega 56) and ended up running FaH under Windows 10 for Workstations because it just kept failing to find OpenCL on Linux. I pulled out my hair for days, trying the FOSS opencl-icd as well as the (finally releaed) amdgpu-pro for 18.04.4 - headless *and* full - to no avail.

With your systemd service file I have now got everything running properly. Again, thanks! :)
Rainmaker
 
Posts: 15
Joined: Sat Mar 28, 2020 6:59 pm

Re: HOWTO: How I got my R9 290 folding on Linux

Postby Juanro49 » Sat Mar 28, 2020 8:45 pm

Frogging101 wrote:My setup is as follows:

Code: Select all
Radeon R9 290 (Hawaii) with free software drivers (amdgpu/mesa)
Intel Core i7-4790
Linux Mint 18.1 64-bit
Linux kernel v5.5.1


I created this guide to share my findings and experience in setting up Folding@Home on this computer. I hope that it can provide some insight and help to others.

Installing the AMDGPU-PRO OpenCL implementation

I use the free software GPU driver stack. Unfortunately, I could not get Folding@Home to work with the free software OpenCL implementations from ROCm or Mesa. So I will not bother explaining how to set those up. If you have them, you may want to remove them in case they interfere.

What did work was something suggested by dschuermann on IRC (#radeon on freenode; great people there). This allows you to use the OpenCL implementation from the proprietary AMDGPU-PRO driver, without installing the whole driver.

  1. Get the AMDGPU-PRO driver package:
    1. Go to https://www.amd.com/en/support and select your graphics card.
    2. Scroll down to Ubuntu x86 64-bit and download the file. (currently: amdgpu-pro-19.50-967956-ubuntu-18.04.tar.xz)
    3. Extract it to an empty directory somewhere. Do not run the install script or anything else.
  2. Extract the OpenCL driver from the appropriate .deb file.
    For Vega 10 and later products, this is the .deb file matching the pattern opencl-amdgpu-pro-icd_*_amd64.deb.
    For products older than Vega 10, this is the .deb file matching the pattern opencl-orca-amdgpu-pro-icd_*_amd64.deb.
    1. In a terminal in the directory with the .deb files, run the following:
      Code: Select all
      # Extract the contents of the .deb package to the current directory
      dpkg -x DEB_FILE . # replace DEB_FILE with the appropriate pattern from above

      # Replace the path in the .icd file with the full path to where we're going to put the library
      sed -i 's|^|/opt/amdgpu-pro/lib/x86_64-linux-gnu/|' etc/OpenCL/vendors/amdocl*.icd

      # Copy the etc and opt directories extracted from the package to the system root.
      sudo cp -r etc opt /


Fixing the permissions issue (clGetDeviceIDs() returned -1)

Some donors have been getting "OpenCL: Not detected: clGetDeviceIDs() returned -1" unless they run FAHClient as root, and adding the fahclient user to the video group doesn't fix it.

I observed this myself, and upon investigating, I discovered the problem. The official Folding@Home packages (at least the Debian-based ones) set up FAHClient to be launched by an init.d script, which launches FAHClient with "--run-as fahclient", so that it runs as the "fahclient" user.

The problem is that when FAHClient is launched with the --run-as option, it does not give itself the supplementary groups of the user passed into the --run-as option. Thus the FAHClient process will not have the permissions of the groups that the "fahclient" user is a member of. You can see this for yourself:
Code: Select all
$ pidof FAHClient
26801 26796
$ sudo cat /proc/26796/status | grep 'Groups:'
Groups:


The lack of any group numbers after "Groups:" shows that the FAHClient process does not have any supplementary groups, even though it is running as the "fahclient" user. This is due to a bug in FAHClient.

For the developers: If I had to guess (I have not verified this), this is probably happening because FAHClient uses setuid() to drop privileges with the "--run-as" argument. This is not sufficient. You must call initgroups() before setuid() to give the process the supplementary groups of the target user.

TL;DR For the users: To work around this, I created a systemd unit to start FAHClient instead of the init.d script. This works by using the systemd unit's "User" option to change users instead of having FAHClient do it.

Create /etc/systemd/system/FAHClient.service (The name matters; calling it this will automatically supersede the init.d script of the same name), with the following contents:
Code: Select all
[Unit]
Description=Folding@Home Client
After=network.target

[Service]
Type=simple
ExecStart=/usr/bin/FAHClient /etc/fahclient/config.xml --pid-file=/var/run/fahclient/fahclient.pid --daemon
PIDFile=/var/run/fahclient/fahclient.pid
User=fahclient
RuntimeDirectory=fahclient
WorkingDirectory=~

[Install]
WantedBy=multi-user.target


Thanks, it work for me, but I also need to extract opencl-amdgpu-pro-comgr to detect GPU Navi
Consulta como ver TV y leer noticias desde tu movil en una sola app con FeedTV.
Apoya la investigación desde esta imagen Image
Juanro49
 
Posts: 16
Joined: Wed Mar 18, 2020 9:10 pm

Re: HOWTO: How I got my R9 290 folding on Linux

Postby Rainmaker » Sat Mar 28, 2020 11:00 pm

Removed as the advice causes issues for some configurations.
Last edited by Rainmaker on Sun Mar 29, 2020 3:14 pm, edited 1 time in total.
Rainmaker
 
Posts: 15
Joined: Sat Mar 28, 2020 6:59 pm

Re: HOWTO: How I got my R9 290 folding on Linux

Postby Juanro49 » Sun Mar 29, 2020 1:03 pm

Rainmaker wrote:
Juanro49 wrote:Thanks, it work for me, but I also need to extract opencl-amdgpu-pro-comgr to detect GPU Navi


That reminds me...

OP, you don't need to dig out random .deb files to install the AMD OpenCL stuff. In your OP you say that you need/want the free AMD display driver. So do I, for vaapi to work in patched Chromium. If you install the full proprietary AMDGPU/AMDGPU-PRO driver stack Chromium can't use the driver with VAAPI and reverts to software decoding for video.

There is, however, an easier way. You can invoke the AMD installer script but tell it to only install OpenCL.

For Vega and later cards:
Code: Select all
amdgpu-pro-install --headless --opencl=pal


For cards before Vega:
Code: Select all
amd-pro-install --headless --opencl=legacy


The --headless flag tells it to not install the main graphics components. You can also combine opencl options if you needed to (opencl=legacy,pal). This leaves the open graphics stack working OK and installs the opencl stuff. Worked fine for me on Ubuntu 18.04.4. Just offering this in case it helps out you, or someone else.


This option installs some additional software instead of just OpenCL support, I tried this option weeks ago and I had problems viewtopic.php?f=81&t=32831
Juanro49
 
Posts: 16
Joined: Wed Mar 18, 2020 9:10 pm

Re: HOWTO: How I got my R9 290 folding on Linux

Postby Rainmaker » Sun Mar 29, 2020 3:13 pm

Juanro49 wrote:This option installs some additional software instead of just OpenCL support, I tried this option weeks ago and I had problems viewtopic.php?f=81&t=32831


Ah, thanks for the heads up. It worked OK for me but in that case the manual way is better to be safe.
Rainmaker
 
Posts: 15
Joined: Sat Mar 28, 2020 6:59 pm

Re: HOWTO: How I got my R9 290 folding on Linux

Postby Frogging101 » Sun Mar 29, 2020 4:48 pm

Yeah, I didn't use the install script because I don't expect it to be designed for this very specific use case, and I don't trust it to do it properly.
Frogging101
 
Posts: 69
Joined: Wed Mar 25, 2020 3:39 am
Location: Canada

Re: HOWTO: How I got my R9 290 folding on Linux

Postby warrelium » Mon Mar 30, 2020 4:36 am

Hey Frogging101, I installed the proprietary OpenCl. Thank you for the insight. I'm on Ubuntu 18.04.4

However, even after adding fahclient to video group and with your service unit I am unable to get the FAH Client to launch with the permissions needed to get past the (clGetDeviceIDs() returned -1) error.

After creating /etc/systemd/system/FAHClient.service the FAHClient process does not start at boot

If I run FAHClient with -u root then it sees the openCl device and I am able to properly fold.

Is there a different approach I need to take with the FAHClient.service to get it to work on Ubuntu 18.04.4?


EDIT:
Tonight I learned that you have to enable the service to get it to run. The service now launches the FAHClient on boot after I did the following:

Code: Select all
sudo chmod 644 /etc/systemd/system/FAHClient.service

Code: Select all
sudo systemctl enable FAHClient
warrelium
 
Posts: 6
Joined: Mon Mar 30, 2020 4:23 am

Re: HOWTO: How I got my R9 290 folding on Linux

Postby Frogging101 » Mon Mar 30, 2020 6:55 pm

Thanks warrelium. I forgot about that. I updated the OP.
Frogging101
 
Posts: 69
Joined: Wed Mar 25, 2020 3:39 am
Location: Canada

Re: HOWTO: How I got my R9 290 folding on Linux

Postby toTOW » Tue Mar 31, 2020 9:52 pm

Stickyfied topic so that we can refer people to it.
Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.

FAH-Addict : latest news, tests and reviews about Folding@Home project.

Image
User avatar
toTOW
Site Moderator
 
Posts: 5652
Joined: Sun Dec 02, 2007 11:38 am
Location: Bordeaux, France

Re: HOWTO: How I got my R9 290 folding on Linux

Postby Ksenio » Wed Apr 01, 2020 5:07 pm

I tried following this, but I have still not made it work. I am running RX550 and Mint 19.3 . I have installed orca, but I am still getting this error
16:05:49:ERROR:WU00:FS01:Failed to start core: OpenCL device matching slot 1 not found, try setting 'opencl-index' manually

Any ideas what I could explore more? I am really determined to make it work :)
Ksenio
 
Posts: 1
Joined: Wed Apr 01, 2020 5:02 pm

Next

Return to Problems with AMD/ATI drivers

Who is online

Users browsing this forum: Google [Bot] and 3 guests

cron