How to set up Folding@home on FreeNAS/TrueNAS - a guide

FAH provides a V7 client installer for Debian / Mint / Ubuntu / RedHat / CentOS / Fedora. Installation on other distros may or may not be easy but if you can offer help to others, they would appreciate it.

Moderators: Site Moderators, FAHC Science Team

Post Reply
Jesse_V
Site Moderator
Posts: 2851
Joined: Mon Jul 18, 2011 4:44 am
Hardware configuration: OS: Windows 10, Kubuntu 19.04
CPU: i7-6700k
GPU: GTX 970, GTX 1080 TI
RAM: 24 GB DDR4
Location: Western Washington

How to set up Folding@home on FreeNAS/TrueNAS - a guide

Post by Jesse_V »

This is a guide on how to configure a PC to manage a large storage pool using FreeNAS and running Folding@home at the same time. Up until now, to the extent of my research on this forum and across the Internet, they seemed fairly mutually exclusive. There are alternatives, such as Ansible, but FreeNAS is the most popular and widely-recommended OS for a NAS. However, it is based on FreeBSD, which is not supported by Folding@home, and it is not possible to run F@h inside a FreeBSD jail in FreeNAS. Therefore another solution is needed.

First, I tried setting up a lightweight headless Debian VM inside FreeNAS. I installed the Nvidia drivers and set it up for F@h as normal. The BSD hypervisor, bhyve, does support GPU passthrough. However, Phoronix reported that performance is around 66% compared to bare metal. This is not good for F@h and not really acceptable for the high-priority COVID Moonshot sprint projects. I therefore gave up on FreeNAS for F@h at first.

However, over the past few months, iXsystems started unifying the FreeNAS and TrueNAS platforms and announced TrueNAS SCALE as a new project. As TrueNAS SCALE is based on Debian, F@h is supported and I took a closer look at it. Here is how you can get Folding@home up and running on TrueNAS SCALE:
  1. Follow the guides to install TrueNAS onto a USB flashdrive.
  2. Use the web interface to import or create a storage pool and get TrueNAS set up.
  3. Use the terminal interface on the PC to drop into a shell.
  4. Install the Nvidia drivers with:

    Code: Select all

    apt-get --no-install-recommends install nvidia-driver nvidia-opencl-icd nvidia-cuda-mps nvidia-smi nvidia-tesla-440-opencl-icd
    You can also use nvidia-detect to list detected hardware and suggest the correct driver. Its possible that ocl-icd-opencl-dev or nvidia-opencl-dev could be the packages strictly necessary for a minimal system, but the above sets up everything as needed.
  5. Use the web interface to restart the system.
  6. Double-check that the drivers are correctly installed. You can run nvidia-smi to see the status of connected graphics cards. I also installed "clinfo" and ran

    Code: Select all

    clinfo | less
    to run a test of OpenCL and CUDA.
  7. Install FAHClient on the system. As of writing, the FAHClient package for Debian can be found from the download page at specifically with this link. This URL can be fetched with wget and installed with

    Code: Select all

    dpkg -i fahclient*.deb
  8. Follow the initial configuration guide to set up the F@h client for the first time. I suggest selecting "Full" to ensure that the client is set up for all hardware.
  9. Modify the config.xml file at /etc/fahclient/config.xml. Add the following lines:

    Code: Select all

    <allow v='127.0.0.1 192.168.0.0/16'/>
    <password v='myfahpassword/>
    
    The network subnet can be customized for your LAN to allow a connection from FAHControl on another PC. The password should also be changed.
  10. Start F@h with either

    Code: Select all

    systemctl start FAHClient
    or

    Code: Select all

    /etc/init.d/FAHClient start
    I prefer the latter.
  11. Examine the logs with

    Code: Select all

    tail -30 /var/lib/fahclient/log.txt
    and ensure that it is running correctly. The beginning of the log should list detected GPUs and find the OpenCL and CUDA platforms. If workunits are running, especially GPU workunits, you've got it set up correctly!
  12. Connect FAHControl. On another PC, open FAHControl, click "Add" in the bottom-left corner, add the IP address of the NAS and the above password, and click Save. Assuming no firewalls are in the way for TCP/36330, it should connect. This will make it easy to control this instance on the NAS.
  13. Back on the TrueNAS terminal, navigate to the root directory, which should be the default home location for the root user.
  14. Back up the fahclient user with

    Code: Select all

    tail -1 /etc/passwd > fah_user.txt
    This is critical as TrueNAS SCALE seems to regenerate /etc/passwd at boot. Without this entry, FAHClient will not be able to start.
  15. Use the web interface to set up a startup script. As of writing, this can be found at Data Protection > Init/Shutdown Scripts. Create a POSTINIT script that runs the following command:

    Code: Select all

    cat /root/fah_user.txt >> /etc/passwd; /etc/init.d/FAHClient start
  16. Create a SHUTDOWN script that runs the following command to shut down F@h:

    Code: Select all

    /etc/init.d/FAHClient stop
  17. Exit the shell in the TrueNAS terminal to return to the main menu.
  18. Use the web interface to restart the system. From the terminal, see that F@h is shutting down. At startup, you should hear the fans kick up, or check the logs to double-check that F@h is running again. If not, review /etc/passwd to verify that the entry for fahclient is present at the bottom of the file.
  19. Write a F@h monitoring script to run on the TrueNAS terminal screen. I created monitor.sh:

    Code: Select all

    #!/usr/bin/env bash
    
    watch -t -c -p -n 3 " \
        nvidia-smi | grep -v -e '^\s*$' \
        && echo \
        && top -b -n 1 | head -13 \
        && echo \
        && tail -5 /var/lib/fahclient/log.txt"
    
    I also added some lines that used netstat, vnstat, and df to display network activity and disk consumption, but the above works for F@h.
  20. It should be set now!
I am seeing about a 20 to 25% gain in improvement over Windows, which is fairly common for Linux, so I'm happy. Another option is to use a Docker instance. I tried both the official Docker container and a third-party container, and while both recognized the GPUs, they failed to accept the OpenCL drivers or open a port for remote control. I suspect (but did not prove) that this is due to the early development of TrueNAS SCALE, so its something that I might circle back to next year when TrueNAS SCALE is fully released. In any case, installing F@h natively on the system works too. Please post below if this also works for you or if you have any suggestions for this guide.
F@h is now the top computing platform on the planet and nothing unites people like a dedicated fight against a common enemy. This virus affects all of us. Lets end it together.
PantherX
Site Moderator
Posts: 7020
Joined: Wed Dec 23, 2009 9:33 am
Hardware configuration: V7.6.21 -> Multi-purpose 24/7
Windows 10 64-bit
CPU:2/3/4/6 -> Intel i7-6700K
GPU:1 -> Nvidia GTX 1080 Ti
§
Retired:
2x Nvidia GTX 1070
Nvidia GTX 675M
Nvidia GTX 660 Ti
Nvidia GTX 650 SC
Nvidia GTX 260 896 MB SOC
Nvidia 9600GT 1 GB OC
Nvidia 9500M GS
Nvidia 8800GTS 320 MB

Intel Core i7-860
Intel Core i7-3840QM
Intel i3-3240
Intel Core 2 Duo E8200
Intel Core 2 Duo E6550
Intel Core 2 Duo T8300
Intel Pentium E5500
Intel Pentium E5400
Location: Land Of The Long White Cloud
Contact:

Re: How to set up Folding@home on FreeNAS/TrueNAS - a guide

Post by PantherX »

Fantastic write-up, Jesse_V! Thanks for sharing the detailed steps and rational as to why it's needed :eugeek:
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time

Welcome To The F@H Support Forum Ӂ Troubleshooting Bad WUs Ӂ Troubleshooting Server Connectivity Issues
Jesse_V
Site Moderator
Posts: 2851
Joined: Mon Jul 18, 2011 4:44 am
Hardware configuration: OS: Windows 10, Kubuntu 19.04
CPU: i7-6700k
GPU: GTX 970, GTX 1080 TI
RAM: 24 GB DDR4
Location: Western Washington

Re: How to set up Folding@home on FreeNAS/TrueNAS - a guide

Post by Jesse_V »

Caution: the F@h installation, configuration of Docker images, and other changes are specific to the running version of TrueNAS SCALE and do not seem to follow from one version to the next. After an update, the default home directory (/root) seems normal, but the system is using a fresh image, so F@h does not seem to be installed, /etc/fahclient is not found, and Docker reports that no images are available. I need to further test this to see if this is only happening to the version update I just did, or if this is a general issue. I believe it's possible to get these services to stay within /root/ or a ZFS volume, which should survive between updates.
F@h is now the top computing platform on the planet and nothing unites people like a dedicated fight against a common enemy. This virus affects all of us. Lets end it together.
Trevor68
Posts: 1
Joined: Tue Jul 12, 2022 7:11 am

Re: How to set up Folding@home on FreeNAS/TrueNAS - a guide

Post by Trevor68 »

Hi,
The Truecharts guys have added Folding to their catalog for Truenas Scale as an app. It runs very well, but is running my CPU and GPU by default.
I was hoping someone cleverer than I might be able to help me figure out how to remove the CPU slot?
Post Reply