Folding with Tesla K20C

It seems that a lot of GPU problems revolve around specific versions of drivers. Though NVidia has their own support structure, you can often learn from information reported by others who fold.

Moderators: Site Moderators, FAHC Science Team

N0OA
Posts: 38
Joined: Wed Feb 13, 2013 6:55 am
Hardware configuration: CPU:4 AMD Phenom II X4 910 @ 2.6GHz
CPU:2 Intel Core Duo T2600 @ 2.13GHz
CPU:4 Intel Core i5
CPU:4 Intel Core i5 M520 2.40 GHz
CPU:8 Intel Core i7-2600K @ 3.40GHz
CPU:8 Intel Core i7-3720QM @ 2.6GHz
CPU:7 Intel Core i7-3770 @ 3.40GHz
CPU:8 Intel Core i7-3820QM @ 2.7GHz
CPU:12 Intel Core i7-3930K @ 3.20GHz
CPU:10 Intel Core i7-3960X Hexa-Core 3.3GHz
CPU:10 Intel Core i7-3960X Hexa-Core 3.3GHz
CPU:2 Intel Pentium® D @ 2.80GHz
CPU:30 Intel XEON CPU E5-2687W @3.1GHz (2x)
GPU NVIDIA GT 640
GPU NVIDIA GT218 [NVS 3100M]
GPU NVIDIA GTX 570 HD EVGA
GPU NVIDIA GTX 660 Ti Zotac
GPU NVIDIA GTX 660 Ti Zotac
GPU NVIDIA GTX 660 Ti Zotac
GPU NVIDIA GTX 660 Ti Zotac
GPU NVIDIA GTX 680 EVGA
GPU NVIDIA GTX 680 EVGA
GPU NVIDIA GTX 680 GeForce
GPU NVIDIA GTX 680 GeForce
GPU NVIDIA GTX 680 GeForce
GPU NVIDIA GTX 680 GIGABYTE
GPU NVIDIA GTX 680 GIGABYTE
GPU NVIDIA GTX Titan EVGA
GPU NVIDIA GTX Titan EVGA
GPU NVIDIA Tesla K20c
Location: Minnesota

Folding with Tesla K20C

Post by N0OA »

Does anyone have experience folding with the Tesla K20C? How should it be setup and what experience do folks have with its PPD? I am currently getting a 37101 on a 7626 work unit configured as "client-type=BIGADV". Is it configured right? What can I expect for PPD out of this card?

Thanks in advance for any advice or thoughts...

N0OA
7im
Posts: 10189
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:

Re: Folding with Tesla K20C

Post by 7im »

bigadv only applies to CPU work units, not GPU.

Not many have Tesla's and I've seen no PPD numbers posted, but there is this thread showing FAHBench performance against many other GPUs, so you can see relative performance. http://foldingforum.org/viewtopic.php?f=38&t=23440
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
N0OA
Posts: 38
Joined: Wed Feb 13, 2013 6:55 am
Hardware configuration: CPU:4 AMD Phenom II X4 910 @ 2.6GHz
CPU:2 Intel Core Duo T2600 @ 2.13GHz
CPU:4 Intel Core i5
CPU:4 Intel Core i5 M520 2.40 GHz
CPU:8 Intel Core i7-2600K @ 3.40GHz
CPU:8 Intel Core i7-3720QM @ 2.6GHz
CPU:7 Intel Core i7-3770 @ 3.40GHz
CPU:8 Intel Core i7-3820QM @ 2.7GHz
CPU:12 Intel Core i7-3930K @ 3.20GHz
CPU:10 Intel Core i7-3960X Hexa-Core 3.3GHz
CPU:10 Intel Core i7-3960X Hexa-Core 3.3GHz
CPU:2 Intel Pentium® D @ 2.80GHz
CPU:30 Intel XEON CPU E5-2687W @3.1GHz (2x)
GPU NVIDIA GT 640
GPU NVIDIA GT218 [NVS 3100M]
GPU NVIDIA GTX 570 HD EVGA
GPU NVIDIA GTX 660 Ti Zotac
GPU NVIDIA GTX 660 Ti Zotac
GPU NVIDIA GTX 660 Ti Zotac
GPU NVIDIA GTX 660 Ti Zotac
GPU NVIDIA GTX 680 EVGA
GPU NVIDIA GTX 680 EVGA
GPU NVIDIA GTX 680 GeForce
GPU NVIDIA GTX 680 GeForce
GPU NVIDIA GTX 680 GeForce
GPU NVIDIA GTX 680 GIGABYTE
GPU NVIDIA GTX 680 GIGABYTE
GPU NVIDIA GTX Titan EVGA
GPU NVIDIA GTX Titan EVGA
GPU NVIDIA Tesla K20c
Location: Minnesota

Re: Folding with Tesla K20C

Post by N0OA »

Thanks for the client-type correction. I didn't catch that in the V7 documentation. I will take a look at the link to see what to expect. The Tesla isn't the fastest card I have - but it runs very cool and has a nice compact form factor for the performance it seems to return.

-N0OA
Quisarious
Posts: 54
Joined: Thu Dec 13, 2012 6:16 pm

Re: Folding with Tesla K20C

Post by Quisarious »

As far as FAH is concerned, a K20C is a detuned gtx780. There are posted PPD estimates for the 780, just divide tpfs by ~0.65 (K20C runs at ~700 core clock, while the 780 will boost to 1000-1200) to get a good estimate for the tesla.
jaysenw
Posts: 6
Joined: Sat Aug 10, 2013 4:45 pm

Re: Folding with Tesla K20C

Post by jaysenw »

Hello;

I have 2 Tesla C1060's. I have not been able to successfully get them to work on FAH. The Tesla card begins folding, then I eventually stops at 99.99%. I have tried removing and replacing the slot. Deleting work units and restarting. The same problem occurs. Always gets to 99.99% on the GUI, never to the same percentage in the log.

I have attached to most recent snippet of the system log for that slot below. Does anyone know what this issue is and possibly how to fix it?

Code: Select all

07:07:34:WU02:FS01:Cleaning up
07:07:34:WU01:FS01:Connecting to assign-GPU.stanford.edu:80
07:07:34:WU01:FS01:News: Welcome to Folding@Home
07:07:34:WU01:FS01:Assigned to work server 171.67.108.21
07:07:34:WU01:FS01:Requesting new work unit for slot 01: READY gpu:1:GT200 [Tesla C1060] from 171.67.108.21
07:07:34:WU01:FS01:Connecting to 171.67.108.21:8080
07:07:35:WU01:FS01:Downloading 61.92KiB
07:07:35:WU01:FS01:Download complete
07:07:35:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:10501 run:162 clone:1 gen:1340 core:0x11 unit:0x00000b466652eda54b6ea7a700003f4b
07:07:35:WU01:FS01:Starting
07:07:35:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/G80/Core_11.fah/FahCore_11.exe -dir 01 -suffix 01 -version 703 -lifeline 2692 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
07:07:35:WU01:FS01:Started FahCore on PID 1524
07:07:35:WU01:FS01:Core PID:3524
07:07:35:WU01:FS01:FahCore 0x11 started
07:07:36:WU01:FS01:0x11:
07:07:36:WU01:FS01:0x11:*------------------------------*
07:07:36:WU01:FS01:0x11:Folding@Home GPU Core
07:07:36:WU01:FS01:0x11:Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
07:07:36:WU01:FS01:0x11:
07:07:36:WU01:FS01:0x11:Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
07:07:36:WU01:FS01:0x11:Build host: amoeba
07:07:36:WU01:FS01:0x11:Board Type: Nvidia
07:07:36:WU01:FS01:0x11:Core      : 
07:07:36:WU01:FS01:0x11:Preparing to commence simulation
07:07:36:WU01:FS01:0x11:- Looking at optimizations...
07:07:36:WU01:FS01:0x11:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
07:07:36:WU01:FS01:0x11:- Created dyn
07:07:36:WU01:FS01:0x11:- Files status OK
07:07:36:WU01:FS01:0x11:- Expanded 62895 -> 336763 (decompressed 535.4 percent)
07:07:36:WU01:FS01:0x11:Called DecompressByteArray: compressed_data_size=62895 data_size=336763, decompressed_data_size=336763 diff=0
07:07:36:WU01:FS01:0x11:- Digital signature verified
07:07:36:WU01:FS01:0x11:
07:07:36:WU01:FS01:0x11:Project: 10501 (Run 162, Clone 1, Gen 1340)
07:07:36:WU01:FS01:0x11:
07:07:36:WU01:FS01:0x11:Assembly optimizations on if available.
07:07:36:WU01:FS01:0x11:Entering M.D.
07:07:41:WU01:FS01:0x11:Tpr hash 01/wudata_01.tpr:  3117068995 2761589855 2641796126 1345936202 2531672404
07:07:41:WU01:FS01:0x11:
07:07:41:WU01:FS01:0x11:Calling fah_main args: 14 usage=100
07:07:41:WU01:FS01:0x11:
07:07:42:WU01:FS01:0x11:Working on Protein
07:07:43:WU01:FS01:0x11:Client config unavailable.
07:07:43:WU01:FS01:0x11:Starting GUI Server
07:08:48:WU01:FS01:0x11:Completed 1%
07:09:53:WU01:FS01:0x11:Completed 2%
Thanks for any input you may have...


Jaysen
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Folding with Tesla K20C

Post by bruce »

When FAH's control application eventually stops at 99.99% there's a definite problem with your GPU or it's drivers. If you look at the log, you will find that folding stopped before reaching 99.99%. You'll also find that there was a driver reset error logged by Windows (if you're running Windows) at that same time and you may have seen the message. Once a driver reset occurs, FAH cannot continue processing but FAH's Control application continues to (incorrectly) report progress until it reaches 99.99%.

Driver resets are caused by a GPU that has hung. That hang may be due to overclocking, due to overheating, due to flaky drivers or due to defective hardware. You need to figure out what's wrong with your system.
GreyWhiskers
Posts: 660
Joined: Mon Oct 25, 2010 5:57 am
Hardware configuration: a) Main unit
Sandybridge in HAF922 w/200 mm side fan
--i7 2600K@4.2 GHz
--ASUS P8P67 DeluxeB3
--4GB ADATA 1600 RAM
--750W Corsair PS
--2Seagate Hyb 750&500 GB--WD Caviar Black 1TB
--EVGA 660GTX-Ti FTW - Signature 2 GPU@ 1241 Boost
--MSI GTX560Ti @900MHz
--Win7Home64; FAH V7.3.2; 327.23 drivers

b) 2004 HP a475c desktop, 1 core Pent 4 HT@3.2 GHz; Mem 2GB;HDD 160 GB;Zotac GT430PCI@900 MHz
WinXP SP3-32 FAH v7.3.6 301.42 drivers - GPU slot only

c) 2005 Toshiba M45-S551 laptop w/2 GB mem, 160GB HDD;Pent M 740 CPU @ 1.73 GHz
WinXP SP3-32 FAH v7.3.6 [Receiving Core A4 work units]
d) 2011 lappy-15.6"-1920x1080;i7-2860QM,2.5;IC Diamond Thermal Compound;GTX 560M 1,536MB u/c@700;16GB-1333MHz RAM;HDD:500GBHyb w/ 4GB SSD;Win7HomePrem64;320.18 drivers FAH 7.4.2ß
Location: Saratoga, California USA

Re: Folding with Tesla K20C

Post by GreyWhiskers »

If it just stopped because of a GPU/driver reset, the system should be able to recover by rebooting the computer. Upon restart, the FAH software should recover to the last checkpoint. It wouldn't be a bad idea to let the system "rest" for a little time (unspecified duration) until restart if the root cause was thermal.

If this doesn't work, then there was something else going on that caused the client to purge the files.

Was the log snippet posted above the bottom of the log when the FAH software quit? If not, it would be interesting to see from the very end of the log any warnings or errors that had been logged.

07:07:36:WU01:FS01:0x11:Project: 10501 (Run 162, Clone 1, Gen 1340)
jaysenw
Posts: 6
Joined: Sat Aug 10, 2013 4:45 pm

Re: Folding with Tesla K20C

Post by jaysenw »

Hmmm. I have tried the rebooting thing, but it still is unable to recover. I'll install some updated drivers and see what the dealio is. If it IS flaky hardware, do you guys recommend software that I can use to test the load and use of the card? I'm thinking like a CPU torture test but for Tesla's instead...

I'll post when I find out my next step. Thanks for the recommendations so far.

:)

Jaysen
N0OA
Posts: 38
Joined: Wed Feb 13, 2013 6:55 am
Hardware configuration: CPU:4 AMD Phenom II X4 910 @ 2.6GHz
CPU:2 Intel Core Duo T2600 @ 2.13GHz
CPU:4 Intel Core i5
CPU:4 Intel Core i5 M520 2.40 GHz
CPU:8 Intel Core i7-2600K @ 3.40GHz
CPU:8 Intel Core i7-3720QM @ 2.6GHz
CPU:7 Intel Core i7-3770 @ 3.40GHz
CPU:8 Intel Core i7-3820QM @ 2.7GHz
CPU:12 Intel Core i7-3930K @ 3.20GHz
CPU:10 Intel Core i7-3960X Hexa-Core 3.3GHz
CPU:10 Intel Core i7-3960X Hexa-Core 3.3GHz
CPU:2 Intel Pentium® D @ 2.80GHz
CPU:30 Intel XEON CPU E5-2687W @3.1GHz (2x)
GPU NVIDIA GT 640
GPU NVIDIA GT218 [NVS 3100M]
GPU NVIDIA GTX 570 HD EVGA
GPU NVIDIA GTX 660 Ti Zotac
GPU NVIDIA GTX 660 Ti Zotac
GPU NVIDIA GTX 660 Ti Zotac
GPU NVIDIA GTX 660 Ti Zotac
GPU NVIDIA GTX 680 EVGA
GPU NVIDIA GTX 680 EVGA
GPU NVIDIA GTX 680 GeForce
GPU NVIDIA GTX 680 GeForce
GPU NVIDIA GTX 680 GeForce
GPU NVIDIA GTX 680 GIGABYTE
GPU NVIDIA GTX 680 GIGABYTE
GPU NVIDIA GTX Titan EVGA
GPU NVIDIA GTX Titan EVGA
GPU NVIDIA Tesla K20c
Location: Minnesota

Re: Folding with Tesla K20C

Post by N0OA »

Hi Jaysenw,

It looks like from your log that you are downloading Core_11. What client-type do you have defined for your slot with the Tesla in it. I would suggest that you set the client-type to advanced so that you use the core_17 which will run much better on the Tesla cards. I am running the Core_17 on my Tesla K20C without any issues at all.

N0OA
AndyE
Posts: 34
Joined: Tue Mar 19, 2013 10:52 pm

Re: Folding with Tesla K20C

Post by AndyE »

N0OA,
would you mind sharing some perf numbers of your K20C?

thanks,
Andy
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Folding with Tesla K20C

Post by bruce »

That gpu uses the GK110 which really shouldn't be getting assignments for FahCore_11. I would have expected assignments for FahCore_15 prior to setting the client-type to advanced and FahCore_17 after.

You didn't answer my (implied) question: Is this Windows or Linux?

Is the file GPUs.txt present, and if so when was it created?
Joe_H
Site Admin
Posts: 7856
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: Folding with Tesla K20C

Post by Joe_H »

@jaysenw - Could you post the beginning of your log that shows the system configuration, etc.

Your questions are about folding with a Tesla C1060 which is based on a different GPU that the Tesla K20C. It appears from what I read on wikipedia to be based on the same GPU as the GTX 285, and recommendations for settings to use and which projects it will fold well will be different.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
jaysenw
Posts: 6
Joined: Sat Aug 10, 2013 4:45 pm

Re: Folding with Tesla K20C

Post by jaysenw »

Sure thing! One i power down theist em and reboot ill send entire log.

With the hardware recommendations made earlier, I custom built a reducer fan and have been able to get the card balanced at 89 degrees under full load. With this, it folds a work unit in about 65 minutes. Is this a reasonable speed given the processor?

I'll get the logs tomorrow and tell you guys when I get a chance to tommowow after school, no rest fortune wicked med students...
Jesse_V
Site Moderator
Posts: 2851
Joined: Mon Jul 18, 2011 4:44 am
Hardware configuration: OS: Windows 10, Kubuntu 19.04
CPU: i7-6700k
GPU: GTX 970, GTX 1080 TI
RAM: 24 GB DDR4
Location: Western Washington

Re: Folding with Tesla K20C

Post by Jesse_V »

Some projects consist of workunits that take longer than workunits from other projects. This can be due to different protein sizes or complexity, or for other reasons. Points Per Day is often a much more accurate yardstick.
F@h is now the top computing platform on the planet and nothing unites people like a dedicated fight against a common enemy. This virus affects all of us. Lets end it together.
n_w95482
Posts: 66
Joined: Tue May 01, 2012 12:46 am
Hardware configuration: CPU: Ryzen 7 5800X3D

GPU: Radeon RX 6700 XT, Radeon RX 6900 XT
Location: California

Re: Folding with Tesla K20C

Post by n_w95482 »

jaysenw wrote:Sure thing! One i power down theist em and reboot ill send entire log.

With the hardware recommendations made earlier, I custom built a reducer fan and have been able to get the card balanced at 89 degrees under full load. With this, it folds a work unit in about 65 minutes. Is this a reasonable speed given the processor?

I'll get the logs tomorrow and tell you guys when I get a chance to tommowow after school, no rest fortune wicked med students...
Hmm, that's a bit warm for that card to be running. From what I've noticed, Tesla cards are usually underclocked compared to their GeForce equivalent, presumably to maximize stability. That, in turn, should lower temperatures.

How is airflow in the case/around the card? I'd also check to see if the card's heatsink and fan need to be cleaned out, and possibly apply fresh thermal paste.
Folding since December 2003. In memory of my mother, who lost her battle with cancer.

Image
Post Reply