Some information for Kepler GPU Users

It seems that a lot of GPU problems revolve around specific versions of drivers. Though NVidia has their own support structure, you can often learn from information reported by others who fold.

Moderators: Site Moderators, FAHC Science Team

Some information for Kepler GPU Users

Postby psaam0001 » Sun Aug 15, 2021 5:50 am

I do not remember seeing this information posted here. So, I thought that I would share two links of importance concerning the Kepler based GPU support in future Windows Hardware Quality Lab (WHQL) certified drivers, starting with updates that will be released during/after October of 2021:

Support Plan for Kepler Series Cards: https://nvidia.custhelp.com/app/answers ... /related/1

List of Kepler Series GeForce Devices*: https://nvidia.custhelp.com/app/answers ... UyMQ%3D%3D

* I will presume that this may also affect Kepler GPU supporting workstation cards, unless NVidia specifically says otherwise.

Unknown to me at this time, is whether or not these support related changes will be implemented in the Linux drivers or kernels. So, I will have to caution you to start planning to upgrade your folding GPU's on your Linux desktops as well.

Paul
psaam0001
 
Posts: 357
Joined: Mon May 18, 2020 3:02 am
Location: Ruckersville, Virginia, USA

Re: Some information for Kepler GPU Users

Postby FalconFour » Wed Nov 24, 2021 6:03 am

I use 2x nVidia Grid K520 cards (each 2x GF 760 equivalents - which are Kepler) in one of my room-heating folding rigs, running Ubuntu 20.x LTS, and after a series of normal WUs, suddenly I started getting errors on each: "Error compiling program: nvrtc: error: invalid value for --gpu-architecture (-arch)". They had been working normally, then suddenly, they just weren't - about a week ago.

Problem is, they fall back to a broken OpenCL implementation. The OpenCL fallback always piles them all onto the same GPU index, so only 1 of the 4 GPUs was getting work - and that 1 would have 4 WUs running simultaneously. I caught that and necked it down to 1 WU by pausing the others and finishing one at a time - fairly slowly (40% slower).

What gives? Did F@H silently drop support for Kepler as well? This is under Linux and no driver changes were done - they just stopped working right. If I were to guess, not being able to see the command line it gives to "nvrtc", I'd suspect it's using a new compile flag that only supports newer GPUs to give them the advantage. Great, understandable - but how about recognizing setups that don't support it, and keeping them back on the old flags?

They're old cards, but the focus of my folding is to use old hardware to contribute useful work while heating my room. No sense buying new hardware for that. It'd be preferable to keep CUDA working so these are maximally utilized, but even just having the OpenCL fall-back working properly would be a start. The nVidia drivers don't seem to play nice with F@H client, so at startup it gives the error "OpenCL: Not detected: Failed to open dynamic library 'libOpenCL.so': libOpenCL.so: cannot open shared object file: No such file or directory" during enumeration of GPUs -- yet it still folds using the GPU when it runs the fallback.

Due to the complete lack of documentation on GPU slot flags, I also can't find a way to get the slots to map to GPUs in the OpenCL flags given to the WU processes. It just sends "-1" and assumes they'll figure it out (they don't, lol).
FalconFour
 
Posts: 24
Joined: Fri Sep 05, 2008 12:57 pm

Re: Some information for Kepler GPU Users

Postby Neil-B » Wed Nov 24, 2021 8:24 am

FalconFour wrote:What gives? Did F@H silently drop support for Kepler as well?


https://foldingforum.org/viewtopic.php?f=24&t=37545
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W10-Pro, RTX3070

(Green/Bold = Active)
Neil-B
 
Posts: 1937
Joined: Sun Mar 22, 2020 6:52 pm
Location: UK

Re: Some information for Kepler GPU Users

Postby FalconFour » Wed Nov 24, 2021 10:33 am



craaaaaaaaap :cry:

welp it was a nice run I suppose. My just-as-old AMD Radeon R9 280x card in my MacPro1,1 continues to fold though :lol: for now!

Getting OpenCL fallback working properly (for multi-multi-GPU systems) would be nice though, but at a 40% handicap on already-slow cards, it honestly struggles to complete WUs in a timely manner as it is. It was doing well under CUDA so that's a shame. :/

Oh well, I'm blasting points off like a rocket with a 3060ti in the living room PC though 8-)
FalconFour
 
Posts: 24
Joined: Fri Sep 05, 2008 12:57 pm

Re: Some information for Kepler GPU Users

Postby Neil-B » Wed Nov 24, 2021 1:28 pm

FalconFour wrote:Oh well, I'm blasting points off like a rocket with a 3060ti in the living room PC though 8-)


What you should be gaining in throughput/contribution to science on your RTX3060Ti with the latest core may well offset the loss of throughput/contribution to science of the older kit ... it may however not resolve any loss space heater issues :(
Neil-B
 
Posts: 1937
Joined: Sun Mar 22, 2020 6:52 pm
Location: UK

Re: Some information for Kepler GPU Users

Postby psaam0001 » Wed Nov 24, 2021 4:08 pm

Neil-B wrote:What you should be gaining in throughput/contribution to science on your RTX3060Ti with the latest core may well offset the loss of throughput/contribution to science of the older kit ... it may however not resolve any loss space heater issues :(


I know I had better get ready to add some Noctua, Fractal or Corsair case fans to my folding rig's, once I am able to do the upgrades.... Hmmm, I may not need to turn the heat on in my room afterwards.

Paul
psaam0001
 
Posts: 357
Joined: Mon May 18, 2020 3:02 am
Location: Ruckersville, Virginia, USA

Re: Some information for Kepler GPU Users

Postby Neil-B » Wed Nov 24, 2021 7:23 pm

psaam0001 wrote:I know I had better get ready to add some Noctua, Fractal or Corsair case fans to my folding rig's, once I am able to do the upgrades.... Hmmm, I may not need to turn the heat on in my room afterwards.

Paul


My case is a Lian Li Air ... It has 12x120mm and 2x80mm managing airflow through case - cpu cooler is an 360mm AIO using three of the 120mm fans - the rtx3070 is a trifan Asus Rog Stric OC so three more fans there - and the psu is a corsair HX1000i so that is a 140mm fan ... and yes that does allow me to OC the gpu to 2025 and keep the office nice and warm ... want to add an RTX3080Ti LC but need the 2nd mortgage before I can get that ;)
Neil-B
 
Posts: 1937
Joined: Sun Mar 22, 2020 6:52 pm
Location: UK

Re: Some information for Kepler GPU Users

Postby psaam0001 » Wed Nov 24, 2021 9:19 pm

My current FAH Hardware is encased in Fractal Core 1000 cases (micro-ATX)... The one I'm running the GTX 1050ti and GTX 1650 Super (this GPU is destined for the e-cycling bin) in, will be moved into a Cooler Master HAF XB EVO Lan/test bench case. As soon as I am fully employed, will be when the upgrades happen.

Paul
psaam0001
 
Posts: 357
Joined: Mon May 18, 2020 3:02 am
Location: Ruckersville, Virginia, USA

Re: Some information for Kepler GPU Users

Postby FalconFour » Fri Nov 26, 2021 12:05 am

holy crap if any of y'all are thinking of e-wasting (read: sending to the grinder) any full size nVidia GPUs newer than Kepler, send it my way... just retired my GRID cards because of aforementioned issues. But if it's still new enough to fold on, geez, cmon. One of my rigs (the GRID system) doesn't even have a case, it's just a motherboard, PSU, cards on a table :lol:
FalconFour
 
Posts: 24
Joined: Fri Sep 05, 2008 12:57 pm

Re: Some information for Kepler GPU Users

Postby psaam0001 » Fri Nov 26, 2021 2:04 am

It will be awhile before I can e-cycle that 1650 Super... The GTX 1050ti is likely going to move to the system that I only have a 450 watt PSU on for now, once I swap in a Ryzen 9 5950X in place of the Ryzen 3 3200G.

I'm starting to think of building an inexpensive system to put the good GT 1030, and soon to be repaired GT 1030 on. No definite plans on that though, as I'm trying to get a car (so I can work to pay for my 'hobby'). Waiting for the supply chain issues in the US to clear up is also a factor.

Paul
psaam0001
 
Posts: 357
Joined: Mon May 18, 2020 3:02 am
Location: Ruckersville, Virginia, USA


Return to Problems with NVidia drivers

Who is online

Users browsing this forum: No registered users and 3 guests

cron