GTX 1080 Bricked while folding? :(

It seems that a lot of GPU problems revolve around specific versions of drivers. Though NVidia has their own support structure, you can often learn from information reported by others who fold.

Moderators: Site Moderators, FAHC Science Team

GTX 1080 Bricked while folding? :(

Postby atomey » Sun Apr 19, 2020 5:22 pm

My main desktop, among other computers, has been folding for CPU and GPU workloads. Early this morning I woke up and heard clicking sounds coming from my computer. After some investigation, I realized my audio interface was looping on and off because of a reboot loop caused by a DPC watch dog violation Blue Screen. The culprit is the Nvidia display driver.

Since around 3 AM this was happening for me. I had my Folding@Home client configured for light workload and there didn't appear to be any cooling issues.

Now I cannot use the 1080 GTX with the Nvidia drivers without the system eventually crashing from system interrupts, which mostly choke the CPU. If I disable the GTX in the device manager and use the integrated graphics, I have no issues.

The GPU still powers on and gives a video signal and even seems to work in 2D sometimes. Once you try to use an Nvidia driver with it, however, the fans go full blast and I can hear some coil whine noises. The GPU still looks brand new and has no visible defects I can see. I highly doubt it was a cooling problem and I'm using a brand new power supply, so I doubt it is a power issue. I reseated the GPU also but no luck.

This is an EVGA 1080 GTX ACX 3.0 GPU if it matters, running Windows 10 Pro 64 bit with an Intel i7 8086k.

The GPU is about 4 years old almost now and I wanted to replace it soon anyways but if I get a 2080 TI I'm concerned Folding@Home will brick my new GPU also... If I can salvage this thing or find the culprit, it would be nice.
atomey
 
Posts: 4
Joined: Sun Apr 19, 2020 5:11 pm

Re: GTX 1080 Bricked while folding? :(

Postby HaloJones » Sun Apr 19, 2020 7:30 pm

Cards will eventually fail whether it be because they're running FAH or Doom. If the Nvidia driver causes it to die, then you're outta luck. I feel your pain.

Personally, I choose to never use factory cooling solutions and have all my cards water-cooled either with custom or with AIO cooling. That puts my temperatures down in the 40s or at worst very low 50s.

Factory cooling typically puts gpus up at 80C or even higher and that for me is way too hot for extended use which is what is normally the case with FAH.
1x Titan X, 5x 1070, 1x 970, 1 x Ryzen 3600

Image
HaloJones
 
Posts: 806
Joined: Thu Jul 24, 2008 11:16 am

Re: GTX 1080 Bricked while folding? :(

Postby Neil-B » Sun Apr 19, 2020 7:37 pm

I feel for you with the card issue … but I am slightly confused … you indicate you client was set for light loads … iirc the GPU is paused and doesn't fold when set for light?
1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent, Quadro K420 1GB, FAH 7.6.13
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro, Quadro M1000M 2GB, FAH 7.6.13
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro, GTX 750Ti 2GB, FAH 7.6.13
Neil-B
 
Posts: 1171
Joined: Sun Mar 22, 2020 6:52 pm
Location: UK

Re: GTX 1080 Bricked while folding? :(

Postby Jan » Sun Apr 19, 2020 8:45 pm

Neil-B wrote:I feel for you with the card issue … but I am slightly confused … you indicate you client was set for light loads … iirc the GPU is paused and doesn't fold when set for light?

Correct. The GPU probably died while not working for FAH. Still, everything HaloJones said applies.
Jan
 
Posts: 80
Joined: Tue Mar 31, 2020 7:46 pm

Re: GTX 1080 Bricked while folding? :(

Postby atomey » Sun Apr 19, 2020 9:33 pm

HaloJones wrote:Cards will eventually fail whether it be because they're running FAH or Doom. If the Nvidia driver causes it to die, then you're outta luck. I feel your pain.

Personally, I choose to never use factory cooling solutions and have all my cards water-cooled either with custom or with AIO cooling. That puts my temperatures down in the 40s or at worst very low 50s.

Factory cooling typically puts gpus up at 80C or even higher and that for me is way too hot for extended use which is what is normally the case with FAH.


Understood, my 8086k is already water cooled with a Corsair kit with pretty low temps (though I've never had a CPU failure).

Have you had any GPU failures with watercooling? Any suggestions for a 2080 TI? I got a replacement EVGA XC Ultra 2080 TI, but it is air-cooled. I would imagine you can get a separate block to replace the built-in heatsink but perhaps I need to get a different card.


Jan wrote:
Neil-B wrote:I feel for you with the card issue … but I am slightly confused … you indicate you client was set for light loads … iirc the GPU is paused and doesn't fold when set for light?

Correct. The GPU probably died while not working for FAH. Still, everything HaloJones said applies.


Ok, that makes me relieved. I've seen other anecdotes of EVGA 1080s failing after 3-4 years so perhaps that's the lifespan.
atomey
 
Posts: 4
Joined: Sun Apr 19, 2020 5:11 pm

Re: GTX 1080 Bricked while folding? :(

Postby Nathan_P » Sun Apr 19, 2020 9:33 pm

I've bricked several cards over the years, a GTS 250, A GTX 275/250 combo card, a 1070Ti, 8800GTX, 2 x 1080 to name but a few. I've also bricked a couple of cards that never saw a WU in their life. They fail, sometimes quickly, sometimes they go on for years. Not sure on EVGA's warranty so look into that before spending on a new card. If you do need a new one, get one with a good warranty or at least from a decent retailer that will cover the warranty for you
Image
Nathan_P
 
Posts: 1170
Joined: Wed Apr 01, 2009 10:22 pm
Location: Jersey, Channel islands

Re: GTX 1080 Bricked while folding? :(

Postby HaloJones » Sun Apr 19, 2020 9:46 pm

https://www.ekwb.com/shop/ek-fc-rtx-208 ... ckel-plexi will fit your GPU but of course you also need tubes, a pump, a reservoir and a radiator. Whereabouts are you?

I have custom cooling on four 1070s and a TitanX but one of the 1070s is cooled with a NZXT G12 and a 280mm Corsair AIO but I wouldn't recommend that for a 2080ti.

Either do full custom which is going to cost you maybe $300 at least or you could set a lower performance profile when folding by using something like Afterburner to set a power limit of maybe 80%?
HaloJones
 
Posts: 806
Joined: Thu Jul 24, 2008 11:16 am

Re: GTX 1080 Bricked while folding? :(

Postby HaloJones » Sun Apr 19, 2020 9:48 pm

Oh, and yes, I've lost a lot of cards over the years to folding. A TitanX, at least one 980ti, numerous 970s; most weren't water-cooled although some where. It really all depends on the card's cooling system. I had a 1070ti that I ran with air because it had fantastic cooling and with the fan set to 80% which I still couldn't hear it ran in the low 60s centigrade.
HaloJones
 
Posts: 806
Joined: Thu Jul 24, 2008 11:16 am

Re: GTX 1080 Bricked while folding? :(

Postby kiore » Sun Apr 19, 2020 9:50 pm

I have lost fans on cards, a high range GTX1080ti hybrid card fan just seized after 8 months of albeit heavy use, entire card replaced by manufacturer with one that lasted considerably longer and still going strong.
For would be RTX 2080ti buyers I regret my twin fan air cooled version, this a very hot card and I find myself in the strange position of power limiting it as a matter of course for folding use to 85% and yes I have extra fans and a roomy case, and yes I tried taking the sides off, it actually works slightly better with sides on and 4 x 140mm case fans going.
Image
i7 7800x GTX 1080ti GTX1660ti, OS= win10. AMD 3700x RTX 2080ti OS= win10

Team page: http://www.rationalskepticism.org/gener ... -t616.html
kiore
 
Posts: 827
Joined: Fri Jan 16, 2009 6:45 pm
Location: USA

Re: GTX 1080 Bricked while folding? :(

Postby atomey » Sun Apr 19, 2020 10:08 pm

HaloJones wrote:https://www.ekwb.com/shop/ek-fc-rtx-2080-ti-classic-rgb-nickel-plexi will fit your GPU but of course you also need tubes, a pump, a reservoir and a radiator. Whereabouts are you?

I have custom cooling on four 1070s and a TitanX but one of the 1070s is cooled with a NZXT G12 and a 280mm Corsair AIO but I wouldn't recommend that for a 2080ti.

Either do full custom which is going to cost you maybe $300 at least or you could set a lower performance profile when folding by using something like Afterburner to set a power limit of maybe 80%?


That block seems pretty cheap, at least compared to the official EVGA one here: https://www.evga.com/products/product.a ... HC-1189-B1

I'm out in Los Angeles but $300 seems reasonable to protect a $1200 investment or at least extend lifespan. If i get a whole kit with the tubes, pump, reservoir, radiator, I would imagine its interchangeable with other blocks, no?

kiore wrote:I have lost fans on cards, a high range GTX1080ti hybrid card fan just seized after 8 months of albeit heavy use, entire card replaced by manufacturer with one that lasted considerably longer and still going strong.
For would be RTX 2080ti buyers I regret my twin fan air cooled version, this a very hot card and I find myself in the strange position of power limiting it as a matter of course for folding use to 85% and yes I have extra fans and a roomy case, and yes I tried taking the sides off, it actually works slightly better with sides on and 4 x 140mm case fans going.

What are your ambient air temps? That's always a factor too, even with the best air cooling.

The 2080 TI i just got seems to be around 46 C idle and 82C with load
atomey
 
Posts: 4
Joined: Sun Apr 19, 2020 5:11 pm

Re: GTX 1080 Bricked while folding? :(

Postby HaloJones » Sun Apr 19, 2020 10:40 pm

atomey wrote:That block seems pretty cheap, at least compared to the official EVGA one here: https://www.evga.com/products/product.a ... HC-1189-B1

I'm out in Los Angeles but $300 seems reasonable to protect a $1200 investment or at least extend lifespan. If i get a whole kit with the tubes, pump, reservoir, radiator, I would imagine its interchangeable with other blocks, no?



The block I linked is made by the largest manufacturer of water-cooling parts. EK are very reputable. I don't know where Tustin is in California but the micro center there has

https://www.microcenter.com/product/391 ... -reservoir - $23
https://www.microcenter.com/product/391 ... 3-8-tubing - $29
https://www.microcenter.com/product/513 ... g-radiator - $69
You will need 3/8" ID tubing to work with that pump and 3/8" OD Barbs with clamps for the radiator and the gpu block so four barbs and clamps.

No idea where you can buy that EK gpu block in the US. I'm in London, UK so not very local to you.

The parts listed were to keep the price down for you. I don't necessarily recommend them but they will work.
HaloJones
 
Posts: 806
Joined: Thu Jul 24, 2008 11:16 am

Re: GTX 1080 Bricked while folding? :(

Postby atomey » Sun Apr 19, 2020 10:51 pm

HaloJones wrote:
atomey wrote:That block seems pretty cheap, at least compared to the official EVGA one here: https://www.evga.com/products/product.a ... HC-1189-B1

I'm out in Los Angeles but $300 seems reasonable to protect a $1200 investment or at least extend lifespan. If i get a whole kit with the tubes, pump, reservoir, radiator, I would imagine its interchangeable with other blocks, no?



The block I linked is made by the largest manufacturer of water-cooling parts. EK are very reputable. I don't know where Tustin is in California but the micro center there has

https://www.microcenter.com/product/391 ... -reservoir - $23
https://www.microcenter.com/product/391 ... 3-8-tubing - $29
https://www.microcenter.com/product/513 ... g-radiator - $69
You will need 3/8" ID tubing to work with that pump and 3/8" OD Barbs with clamps for the radiator and the gpu block so four barbs and clamps.

No idea where you can buy that EK gpu block in the US. I'm in London, UK so not very local to you.

The parts listed were to keep the price down for you. I don't necessarily recommend them but they will work.


Unfortunately no MIcrocenters near LA but I'll order online regardless.

I think I get the idea. I did the configurator on the EK site and ended up with $600+ in parts but I was going aggressive on the build. I think I'll start with something more modest and will definitely use EK. My case already has a 360 radiator unfortunately and only room for another 240 at most.

Anyways thanks for the info!
atomey
 
Posts: 4
Joined: Sun Apr 19, 2020 5:11 pm

Re: GTX 1080 Bricked while folding? :(

Postby HaloJones » Sun Apr 19, 2020 10:56 pm

The 82C you're hitting is what NVidia think is a good max temp. You can use Afterburner to increase the fan speed to a level that doesn't hurt your ears but improves the cooling.
HaloJones
 
Posts: 806
Joined: Thu Jul 24, 2008 11:16 am

Re: GTX 1080 Bricked while folding? :(

Postby kiore » Sun Apr 19, 2020 11:41 pm

I understand that NVIDIA may think 82c a reasonable temp, but I prefer to keep temps below 75c to allow for spikes higher that that I have run burning hot cards at 100% fans before but prefer not to do this is my sleeping area.
kiore
 
Posts: 827
Joined: Fri Jan 16, 2009 6:45 pm
Location: USA

Re: GTX 1080 Bricked while folding? :(

Postby Darth_Peter_dualxeon » Mon Apr 20, 2020 10:03 am

Well, if I were in your place, I'd test that card with a clean freshly installed system and driver and some stress test.
(easy to do if you have a spare disk to fool around.)
it's still not known if the driver or other softwares went stupid or the card is dead.

By the way, if the card is really dead, I'd recommend the gpu type I have right now, an MSI Armor 8G RTX 2070 instead, it's in comparable price range as second-hand GTX 1080. (and if you are lucky, it can be bought second-hand too)
https://www.msi.com/Graphics-card/GeFor ... cification
That has been folding on full load since a month without problem, never goes above 65 Celsius,only 175 watts, and it has a huge cooler and case has good airflow, but it barely fit into the case due to the taller heatsink size)

Or I'd recommend anything with < 200W TDP and 2-3 fans (not the blower fan version, my Tesla C2050 was running at 99C at full load at 250W when used, although that's the industrial version that did survive this)
Darth_Peter_dualxeon
 
Posts: 46
Joined: Fri Mar 20, 2020 4:13 am

Next

Return to Problems with NVidia drivers

Who is online

Users browsing this forum: No registered users and 2 guests

cron