Has my GTX 560 Ti failed [solved: clockrates]

It seems that a lot of GPU problems revolve around specific versions of drivers. Though NVidia has their own support structure, you can often learn from information reported by others who fold.

Moderators: Site Moderators, FAHC Science Team

Post Reply
robintrade
Posts: 4
Joined: Tue Jun 21, 2011 2:13 pm

Has my GTX 560 Ti failed [solved: clockrates]

Post by robintrade »

I'm running a Phenom x6 1090T with a GTX 560 Ti card. The gpu keeps failing. exception thrown in GuardedRun -- cannot continue further.
I have a fresh install of Win 7 HP
I have tried 3 different drivers from nvidia
Currently using 314.xx
Sometimes the gpu will fail after a few minutes of folding, sometimes several hours later.

Any help is appreciated.

Here is the log:

Code: Select all

*********************** Log Started 2013-08-10T06:49:55Z ***********************
06:49:56:WU01:FS00:Connecting to assign-GPU.stanford.edu:80
06:49:57:WU01:FS00:News: Welcome to Folding@Home
06:49:57:WU01:FS00:Assigned to work server 171.64.65.105
06:49:57:WU01:FS00:Requesting new work unit for slot 00: READY gpu:0:GF114 [GeForce GTX 560 Ti] from 171.64.65.105
06:49:57:WU01:FS00:Connecting to 171.64.65.105:8080
06:49:58:WU01:FS00:Downloading 78.86KiB
06:49:58:WU01:FS00:Download complete
06:49:58:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:7660 run:482 clone:0 gen:325 core:0x15 unit:0x00000174664f2dd150f83f6a1a812b75
06:49:58:WU01:FS00:Starting
06:49:58:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Phenom x6/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 01 -suffix 01 -version 703 -lifeline 3708 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
06:49:58:WU01:FS00:Started FahCore on PID 3096
06:49:59:WU01:FS00:Core PID:3264
06:49:59:WU01:FS00:FahCore 0x15 started
06:49:59:WU01:FS00:0x15:
06:49:59:WU01:FS00:0x15:*------------------------------*
06:49:59:WU01:FS00:0x15:Folding@Home GPU Core
06:49:59:WU01:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
06:49:59:WU01:FS00:0x15:Build host             AmoebaRemote
06:49:59:WU01:FS00:0x15:Board Type             NVIDIA/CUDA
06:49:59:WU01:FS00:0x15:Core                   15
06:49:59:WU01:FS00:0x15:
06:49:59:WU01:FS00:0x15:Window's signal control handler registered.
06:49:59:WU01:FS00:0x15:Preparing to commence simulation
06:49:59:WU01:FS00:0x15:- Looking at optimizations...
06:49:59:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
06:49:59:WU01:FS00:0x15:- Created dyn
06:49:59:WU01:FS00:0x15:- Files status OK
06:49:59:WU01:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
06:49:59:WU01:FS00:0x15:- Expanded 80239 -> 307810 (decompressed 383.6 percent)
06:49:59:WU01:FS00:0x15:Called DecompressByteArray: compressed_data_size=80239 data_size=307810, decompressed_data_size=307810 diff=0
06:49:59:WU01:FS00:0x15:- Digital signature verified
06:49:59:WU01:FS00:0x15:
06:49:59:WU01:FS00:0x15:Project: 7660 (Run 482, Clone 0, Gen 325)
06:49:59:WU01:FS00:0x15:
06:49:59:WU01:FS00:0x15:Assembly optimizations on if available.
06:49:59:WU01:FS00:0x15:Entering M.D.
06:50:01:WU01:FS00:0x15:Tpr hash 01/wudata_01.tpr:  4066819132 734559585 3972492986 632110462 1829666135
06:50:01:WU01:FS00:0x15:GPU device id=0
06:50:01:WU01:FS00:0x15:Working on Protein
06:50:01:WU01:FS00:0x15:Client config unavailable.
06:50:01:WU01:FS00:0x15:Starting GUI Server
06:51:07:WU01:FS00:0x15:Setting checkpoint frequency: 400000
06:51:07:WU01:FS00:0x15:Completed         3 out of 40000000 steps (0%).
06:53:41:WU01:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
06:56:15:WU01:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
06:58:50:WU01:FS00:0x15:Completed   1200000 out of 40000000 steps (3%).
07:00:33:WU01:FS00:0x15:Run: exception thrown in GuardedRun -- cannot continue further.
07:00:33:WU01:FS00:0x15:Going to send back what have done -- stepsTotalG=40000000
07:00:33:WU01:FS00:0x15:Work fraction=0.0366 steps=40000000.
07:00:36:WU01:FS00:0x15:logfile size=0 infoLength=0 edr=0 trr=23
07:00:36:WU01:FS00:0x15:+ Opened results file
07:00:36:WU01:FS00:0x15:- Writing 639 bytes of core data to disk...
07:00:36:WU01:FS00:0x15:Done: 127 -> 124 (compressed to 97.6 percent)
07:00:36:WU01:FS00:0x15:  ... Done.
07:00:36:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
07:00:36:WU01:FS00:0x15:
07:00:36:WU01:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
07:00:37:WARNING:WU01:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
07:00:37:WU01:FS00:Sending unit results: id:01 state:SEND error:FAULTY project:7660 run:482 clone:0 gen:325 core:0x15 unit:0x00000174664f2dd150f83f6a1a812b75
07:00:37:WU01:FS00:Uploading 636B to 171.64.65.105
07:00:37:WU01:FS00:Connecting to 171.64.65.105:8080
07:00:37:WU01:FS00:Upload complete
07:00:37:WU01:FS00:Server responded WORK_ACK (400)
07:00:37:WU01:FS00:Cleaning up
07:00:37:WU02:FS00:Connecting to assign-GPU.stanford.edu:80
07:00:38:WU02:FS00:News: Welcome to Folding@Home
07:00:38:WU02:FS00:Assigned to work server 171.64.65.105
07:00:38:WU02:FS00:Requesting new work unit for slot 00: READY gpu:0:GF114 [GeForce GTX 560 Ti] from 171.64.65.105
07:00:38:WU02:FS00:Connecting to 171.64.65.105:8080
07:00:38:WU02:FS00:Downloading 123.59KiB
07:00:39:WU02:FS00:Download complete
07:00:39:WU02:FS00:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:7626 run:154 clone:0 gen:167 core:0x15 unit:0x000000ca664f2dd14fe61aa0e4aa21d4
07:00:39:WU02:FS00:Starting
07:00:39:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Phenom x6/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 02 -suffix 01 -version 703 -lifeline 3708 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
07:00:39:WU02:FS00:Started FahCore on PID 4484
07:00:39:WU02:FS00:Core PID:852
07:00:39:WU02:FS00:FahCore 0x15 started
07:00:39:WU02:FS00:0x15:
07:00:39:WU02:FS00:0x15:*------------------------------*
07:00:39:WU02:FS00:0x15:Folding@Home GPU Core
07:00:39:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
07:00:39:WU02:FS00:0x15:Build host             AmoebaRemote
07:00:39:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
07:00:39:WU02:FS00:0x15:Core                   15
07:00:39:WU02:FS00:0x15:
07:00:39:WU02:FS00:0x15:Window's signal control handler registered.
07:00:39:WU02:FS00:0x15:Preparing to commence simulation
07:00:39:WU02:FS00:0x15:- Looking at optimizations...
07:00:39:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
07:00:39:WU02:FS00:0x15:- Created dyn
07:00:39:WU02:FS00:0x15:- Files status OK
07:00:39:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
07:00:39:WU02:FS00:0x15:- Expanded 126048 -> 507182 (decompressed 402.3 percent)
07:00:39:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=126048 data_size=507182, decompressed_data_size=507182 diff=0
07:00:39:WU02:FS00:0x15:- Digital signature verified
07:00:39:WU02:FS00:0x15:
07:00:39:WU02:FS00:0x15:Project: 7626 (Run 154, Clone 0, Gen 167)
07:00:39:WU02:FS00:0x15:
07:00:39:WU02:FS00:0x15:Assembly optimizations on if available.
07:00:39:WU02:FS00:0x15:Entering M.D.
07:00:41:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  3538847989 3633995712 3158533932 1004457034 2006013503
07:00:41:WU02:FS00:0x15:GPU device id=0
07:00:41:WU02:FS00:0x15:Working on Protein
07:00:41:WU02:FS00:0x15:Client config unavailable.
07:00:41:WU02:FS00:0x15:Starting GUI Server
07:01:47:WU02:FS00:0x15:Setting checkpoint frequency: 400000
07:01:47:WU02:FS00:0x15:Completed         3 out of 40000000 steps (0%).
07:08:14:WU02:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
07:10:28:WU02:FS00:0x15:Run: exception thrown in GuardedRun -- cannot continue further.
07:10:28:WU02:FS00:0x15:Going to send back what have done -- stepsTotalG=40000000
07:10:28:WU02:FS00:0x15:Work fraction=0.0135 steps=40000000.
07:10:31:WU02:FS00:0x15:logfile size=0 infoLength=0 edr=0 trr=23
07:10:31:WU02:FS00:0x15:+ Opened results file
07:10:31:WU02:FS00:0x15:- Writing 639 bytes of core data to disk...
07:10:31:WU02:FS00:0x15:Done: 127 -> 125 (compressed to 98.4 percent)
07:10:31:WU02:FS00:0x15:  ... Done.
07:10:31:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
07:10:31:WU02:FS00:0x15:
07:10:31:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
07:10:32:WARNING:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
07:10:32:WU02:FS00:Sending unit results: id:02 state:SEND error:FAULTY project:7626 run:154 clone:0 gen:167 core:0x15 unit:0x000000ca664f2dd14fe61aa0e4aa21d4
07:10:32:WU02:FS00:Uploading 637B to 171.64.65.105
07:10:32:WU02:FS00:Connecting to 171.64.65.105:8080
07:10:32:WU01:FS00:Connecting to assign-GPU.stanford.edu:80
07:10:32:WU02:FS00:Upload complete
07:10:32:WU02:FS00:Server responded WORK_ACK (400)
07:10:32:WU02:FS00:Cleaning up
07:10:33:WU01:FS00:News: Welcome to Folding@Home
07:10:33:WU01:FS00:Assigned to work server 171.64.65.105
07:10:33:WU01:FS00:Requesting new work unit for slot 00: READY gpu:0:GF114 [GeForce GTX 560 Ti] from 171.64.65.105
07:10:33:WU01:FS00:Connecting to 171.64.65.105:8080
07:10:33:WU01:FS00:Downloading 78.63KiB
07:10:33:WU01:FS00:Download complete
07:10:33:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:7660 run:378 clone:0 gen:347 core:0x15 unit:0x00000197664f2dd150f83f5b69a3d101
07:10:33:WU01:FS00:Starting
07:10:33:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Phenom x6/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 01 -suffix 01 -version 703 -lifeline 3708 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
07:10:33:WU01:FS00:Started FahCore on PID 3456
07:10:33:WU01:FS00:Core PID:2992
07:10:33:WU01:FS00:FahCore 0x15 started
07:10:34:WU01:FS00:0x15:
07:10:34:WU01:FS00:0x15:*------------------------------*
07:10:34:WU01:FS00:0x15:Folding@Home GPU Core
07:10:34:WU01:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
07:10:34:WU01:FS00:0x15:Build host             AmoebaRemote
07:10:34:WU01:FS00:0x15:Board Type             NVIDIA/CUDA
07:10:34:WU01:FS00:0x15:Core                   15
07:10:34:WU01:FS00:0x15:
07:10:34:WU01:FS00:0x15:Window's signal control handler registered.
07:10:34:WU01:FS00:0x15:Preparing to commence simulation
07:10:34:WU01:FS00:0x15:- Looking at optimizations...
07:10:34:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
07:10:34:WU01:FS00:0x15:- Created dyn
07:10:34:WU01:FS00:0x15:- Files status OK
07:10:34:WU01:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
07:10:34:WU01:FS00:0x15:- Expanded 80003 -> 307810 (decompressed 384.7 percent)
07:10:34:WU01:FS00:0x15:Called DecompressByteArray: compressed_data_size=80003 data_size=307810, decompressed_data_size=307810 diff=0
07:10:34:WU01:FS00:0x15:- Digital signature verified
07:10:34:WU01:FS00:0x15:
07:10:34:WU01:FS00:0x15:Project: 7660 (Run 378, Clone 0, Gen 347)
07:10:34:WU01:FS00:0x15:
07:10:34:WU01:FS00:0x15:Assembly optimizations on if available.
07:10:34:WU01:FS00:0x15:Entering M.D.
07:10:36:WU01:FS00:0x15:Tpr hash 01/wudata_01.tpr:  1786857634 1224370871 3670129558 1662232322 2831456867
07:10:36:WU01:FS00:0x15:GPU device id=0
07:10:36:WU01:FS00:0x15:Working on Protein
07:10:36:WU01:FS00:0x15:Client config unavailable.
07:10:36:WU01:FS00:0x15:Starting GUI Server
07:11:41:WU01:FS00:0x15:Setting checkpoint frequency: 400000
07:11:41:WU01:FS00:0x15:Completed         3 out of 40000000 steps (0%).
07:14:15:WU01:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
07:16:50:WU01:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
07:19:24:WU01:FS00:0x15:Completed   1200000 out of 40000000 steps (3%).
07:21:59:WU01:FS00:0x15:Completed   1600000 out of 40000000 steps (4%).
07:24:33:WU01:FS00:0x15:Completed   2000000 out of 40000000 steps (5%).
07:27:08:WU01:FS00:0x15:Completed   2400000 out of 40000000 steps (6%).
07:29:42:WU01:FS00:0x15:Completed   2800000 out of 40000000 steps (7%).
07:32:17:WU01:FS00:0x15:Completed   3200000 out of 40000000 steps (8%).
07:32:41:WU01:FS00:0x15:Run: exception thrown in GuardedRun -- cannot continue further.
07:32:41:WU01:FS00:0x15:Going to send back what have done -- stepsTotalG=40000000
07:32:41:WU01:FS00:0x15:Work fraction=0.0815 steps=40000000.
07:32:45:WU01:FS00:0x15:logfile size=0 infoLength=0 edr=0 trr=23
07:32:45:WU01:FS00:0x15:+ Opened results file
07:32:45:WU01:FS00:0x15:- Writing 639 bytes of core data to disk...
07:32:45:WU01:FS00:0x15:Done: 127 -> 124 (compressed to 97.6 percent)
07:32:45:WU01:FS00:0x15:  ... Done.
07:32:45:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
07:32:45:WU01:FS00:0x15:
07:32:45:WU01:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
07:32:45:WARNING:WU01:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
07:32:45:WU01:FS00:Sending unit results: id:01 state:SEND error:FAULTY project:7660 run:378 clone:0 gen:347 core:0x15 unit:0x00000197664f2dd150f83f5b69a3d101
07:32:45:WU01:FS00:Uploading 636B to 171.64.65.105
07:32:45:WU01:FS00:Connecting to 171.64.65.105:8080
07:32:45:WU02:FS00:Connecting to assign-GPU.stanford.edu:80
07:32:46:WU01:FS00:Upload complete
07:32:46:WU01:FS00:Server responded WORK_ACK (400)
07:32:46:WU01:FS00:Cleaning up
07:32:46:WU02:FS00:News: Welcome to Folding@Home
07:32:46:WU02:FS00:Assigned to work server 171.64.65.105
07:32:46:WU02:FS00:Requesting new work unit for slot 00: READY gpu:0:GF114 [GeForce GTX 560 Ti] from 171.64.65.105
07:32:46:WU02:FS00:Connecting to 171.64.65.105:8080
07:32:46:WU02:FS00:Downloading 124.54KiB
07:32:47:WU02:FS00:Download complete
07:32:47:WU02:FS00:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:7626 run:6 clone:0 gen:282 core:0x15 unit:0x0000018c664f2dd14fe619c76a050f3d
07:32:47:WU02:FS00:Starting
07:32:47:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Phenom x6/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 02 -suffix 01 -version 703 -lifeline 3708 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
07:32:47:WU02:FS00:Started FahCore on PID 3904
07:32:47:WU02:FS00:Core PID:3992
07:32:47:WU02:FS00:FahCore 0x15 started
07:32:47:WU02:FS00:0x15:
07:32:47:WU02:FS00:0x15:*------------------------------*
07:32:47:WU02:FS00:0x15:Folding@Home GPU Core
07:32:47:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
07:32:47:WU02:FS00:0x15:Build host             AmoebaRemote
07:32:47:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
07:32:47:WU02:FS00:0x15:Core                   15
07:32:47:WU02:FS00:0x15:
07:32:47:WU02:FS00:0x15:Window's signal control handler registered.
07:32:47:WU02:FS00:0x15:Preparing to commence simulation
07:32:47:WU02:FS00:0x15:- Looking at optimizations...
07:32:47:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
07:32:47:WU02:FS00:0x15:- Created dyn
07:32:47:WU02:FS00:0x15:- Files status OK
07:32:47:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
07:32:47:WU02:FS00:0x15:- Expanded 127021 -> 507182 (decompressed 399.2 percent)
07:32:47:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=127021 data_size=507182, decompressed_data_size=507182 diff=0
07:32:47:WU02:FS00:0x15:- Digital signature verified
07:32:47:WU02:FS00:0x15:
07:32:47:WU02:FS00:0x15:Project: 7626 (Run 6, Clone 0, Gen 282)
07:32:47:WU02:FS00:0x15:
07:32:47:WU02:FS00:0x15:Assembly optimizations on if available.
07:32:47:WU02:FS00:0x15:Entering M.D.
07:32:49:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  2361582435 811660638 1290256329 909708279 2853598185
07:32:49:WU02:FS00:0x15:GPU device id=0
07:32:49:WU02:FS00:0x15:Working on Protein
07:32:49:WU02:FS00:0x15:Client config unavailable.
07:32:49:WU02:FS00:0x15:Starting GUI Server
07:33:55:WU02:FS00:0x15:Setting checkpoint frequency: 400000
07:33:55:WU02:FS00:0x15:Completed         3 out of 40000000 steps (0%).
07:40:13:WU02:FS00:0x15:Run: exception thrown in GuardedRun -- cannot continue further.
07:40:13:WU02:FS00:0x15:Going to send back what have done -- stepsTotalG=40000000
07:40:13:WU02:FS00:0x15:Work fraction=0.0098 steps=40000000.
07:40:17:WU02:FS00:0x15:logfile size=0 infoLength=0 edr=0 trr=23
07:40:17:WU02:FS00:0x15:+ Opened results file
07:40:17:WU02:FS00:0x15:- Writing 639 bytes of core data to disk...
07:40:17:WU02:FS00:0x15:Done: 127 -> 125 (compressed to 98.4 percent)
07:40:17:WU02:FS00:0x15:  ... Done.
07:40:17:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
07:40:17:WU02:FS00:0x15:
07:40:17:WU02:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
07:40:17:WARNING:WU02:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
07:40:17:WU02:FS00:Sending unit results: id:02 state:SEND error:FAULTY project:7626 run:6 clone:0 gen:282 core:0x15 unit:0x0000018c664f2dd14fe619c76a050f3d
07:40:17:WU02:FS00:Uploading 637B to 171.64.65.105
07:40:17:WU02:FS00:Connecting to 171.64.65.105:8080
07:40:18:WU01:FS00:Connecting to assign-GPU.stanford.edu:80
07:40:18:WU02:FS00:Upload complete
07:40:18:WU02:FS00:Server responded WORK_ACK (400)
07:40:18:WU02:FS00:Cleaning up
07:40:18:WU01:FS00:News: Welcome to Folding@Home
07:40:18:WU01:FS00:Assigned to work server 171.64.65.105
07:40:18:WU01:FS00:Requesting new work unit for slot 00: READY gpu:0:GF114 [GeForce GTX 560 Ti] from 171.64.65.105
07:40:18:WU01:FS00:Connecting to 171.64.65.105:8080
07:40:19:WU01:FS00:Downloading 124.40KiB
07:40:19:WU01:FS00:Download complete
07:40:19:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:7624 run:196 clone:0 gen:266 core:0x15 unit:0x0000015e664f2dd14fe611a7892b51e3
07:40:19:WU01:FS00:Starting
07:40:19:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Phenom x6/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 01 -suffix 01 -version 703 -lifeline 3708 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
07:40:19:WU01:FS00:Started FahCore on PID 4800
07:40:19:WU01:FS00:Core PID:752
07:40:19:WU01:FS00:FahCore 0x15 started
07:40:19:WU01:FS00:0x15:
07:40:19:WU01:FS00:0x15:*------------------------------*
07:40:19:WU01:FS00:0x15:Folding@Home GPU Core
07:40:19:WU01:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
07:40:19:WU01:FS00:0x15:Build host             AmoebaRemote
07:40:19:WU01:FS00:0x15:Board Type             NVIDIA/CUDA
07:40:19:WU01:FS00:0x15:Core                   15
07:40:19:WU01:FS00:0x15:
07:40:19:WU01:FS00:0x15:Window's signal control handler registered.
07:40:19:WU01:FS00:0x15:Preparing to commence simulation
07:40:19:WU01:FS00:0x15:- Looking at optimizations...
07:40:19:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
07:40:19:WU01:FS00:0x15:- Created dyn
07:40:19:WU01:FS00:0x15:- Files status OK
07:40:19:WU01:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
07:40:19:WU01:FS00:0x15:- Expanded 126870 -> 507182 (decompressed 399.7 percent)
07:40:19:WU01:FS00:0x15:Called DecompressByteArray: compressed_data_size=126870 data_size=507182, decompressed_data_size=507182 diff=0
07:40:19:WU01:FS00:0x15:- Digital signature verified
07:40:19:WU01:FS00:0x15:
07:40:19:WU01:FS00:0x15:Project: 7624 (Run 196, Clone 0, Gen 266)
07:40:19:WU01:FS00:0x15:
07:40:19:WU01:FS00:0x15:Assembly optimizations on if available.
07:40:19:WU01:FS00:0x15:Entering M.D.
07:40:21:WU01:FS00:0x15:Tpr hash 01/wudata_01.tpr:  1593859323 3037605081 1239443548 4233789597 2766197497
07:40:21:WU01:FS00:0x15:GPU device id=0
07:40:21:WU01:FS00:0x15:Working on Protein
07:40:21:WU01:FS00:0x15:Client config unavailable.
07:40:21:WU01:FS00:0x15:Starting GUI Server
07:41:27:WU01:FS00:0x15:Setting checkpoint frequency: 400000
07:41:27:WU01:FS00:0x15:Completed         3 out of 40000000 steps (0%).
07:43:56:WU01:FS00:0x15:Run: exception thrown in GuardedRun -- cannot continue further.
07:43:56:WU01:FS00:0x15:Going to send back what have done -- stepsTotalG=40000000
07:43:56:WU01:FS00:0x15:Work fraction=0.0039 steps=40000000.
07:44:00:WU01:FS00:0x15:logfile size=0 infoLength=0 edr=0 trr=23
07:44:00:WU01:FS00:0x15:+ Opened results file
07:44:00:WU01:FS00:0x15:- Writing 639 bytes of core data to disk...
07:44:00:WU01:FS00:0x15:Done: 127 -> 124 (compressed to 97.6 percent)
07:44:00:WU01:FS00:0x15:  ... Done.
07:44:00:WU01:FS00:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
07:44:00:WU01:FS00:0x15:
07:44:00:WU01:FS00:0x15:Folding@home Core Shutdown: UNSTABLE_MACHINE
07:44:00:WARNING:WU01:FS00:FahCore returned: UNSTABLE_MACHINE (122 = 0x7a)
07:44:00:WU01:FS00:Sending unit results: id:01 state:SEND error:FAULTY project:7624 run:196 clone:0 gen:266 core:0x15 unit:0x0000015e664f2dd14fe611a7892b51e3
07:44:00:WU01:FS00:Uploading 636B to 171.64.65.105
07:44:00:WU01:FS00:Connecting to 171.64.65.105:8080
07:44:01:WU01:FS00:Upload complete
07:44:01:WU01:FS00:Server responded WORK_ACK (400)
07:44:01:WU01:FS00:Cleaning up
******************************* Date: 2013-08-10 *******************************
******************************* Date: 2013-08-10 *******************************
******************************* Date: 2013-08-11 *******************************
******************************* Date: 2013-08-11 *******************************
******************************* Date: 2013-08-11 *******************************
Mod edit: Added Code tags around log for you
bollix47
Posts: 2941
Joined: Sun Dec 02, 2007 5:04 am
Location: Canada

Re: Has my GTX 560 Ti failed

Post by bollix47 »

Your frame times suggest that your GPU is overclocked(the reason I suspect this is because I have used a 560ti at stock and your frame times are shorter than mine were). Try changing the memory clock back to stock and if that doesn't work move everything back to stock.

Status 7A

MemtestG80
robintrade
Posts: 4
Joined: Tue Jun 21, 2011 2:13 pm

Re: Has my GTX 560 Ti failed

Post by robintrade »

The card is a Gigabyte GTX 560 Ti Super Over Clock Edition. It is a factory overclock.
The gpu is not folding right now.
Here is a dump file of current status:

########## GPU Observer - Gadget Dump ##########
-----------------------------------------------
OS Version: Microsoft Windows 7 Home Premium SP1 64-bit
RuntimeVersion: v2.0.50727
-----------------------------------------------
ReaderBuild: x86
ReaderValid: True
ReaderVersion: 3.7.0.0
ReaderDirectory: \GPUObserver37.gadget
-----------------------------------------------
NVIDIA Driver: 9.18.13.1422
NVAPI Available: True
-----------------------------------------------
Internal Index: 0
Vendor: NVIDIA
Name: GeForce GTX 560 Ti
GPU Clock: 51MHz
MEM Clock: 135MHz
Shader Clock: 101MHz
GPU Temp: 38°C
PCB Temp: 0°C
GPU Usage: 18%
MCU Usage: 13%
VPU Usage: 0%
FanLevel: 40%
FanRPM: 1770rpm
PCIe: x16
Mem Total: 1024MB
Mem Free: 927MB
-----------------------------------------------
###############################################
robintrade
Posts: 4
Joined: Tue Jun 21, 2011 2:13 pm

Re: Has my GTX 560 Ti failed

Post by robintrade »

And here is the dump after about 1 minute of folding:

########## GPU Observer - Gadget Dump ##########
-----------------------------------------------
OS Version: Microsoft Windows 7 Home Premium SP1 64-bit
RuntimeVersion: v2.0.50727
-----------------------------------------------
ReaderBuild: x86
ReaderValid: True
ReaderVersion: 3.7.0.0
ReaderDirectory: \GPUObserver37.gadget
-----------------------------------------------
NVIDIA Driver: 9.18.13.1422
NVAPI Available: True
-----------------------------------------------
Internal Index: 0
Vendor: NVIDIA
Name: GeForce GTX 560 Ti
GPU Clock: 1000MHz
MEM Clock: 2291MHz
Shader Clock: 2000MHz
GPU Temp: 75°C
PCB Temp: 0°C
GPU Usage: 99%
MCU Usage: 17%
VPU Usage: 0%
FanLevel: 82%
FanRPM: 3330rpm
PCIe: x16
Mem Total: 1024MB
Mem Free: 609MB
-----------------------------------------------
###############################################
bollix47
Posts: 2941
Joined: Sun Dec 02, 2007 5:04 am
Location: Canada

Re: Has my GTX 560 Ti failed

Post by bollix47 »

Pause folding and try dropping the memory clock to 2000. Memory speed has little to no advantage to folding but can cause folding to become unstable if it's too high. You can use MSIAfterburner or EVGA Precision among other software to achieve this.

Start folding again and leave it running. If it fails again the core clock default for that card is 822 and lowering yours from 1000 should lower your shader clock accordingly from 2000 to 1645.

Also you could try running memtestG80 from the link above which should show any problems relating to memory errors.

Another possibility is your GPU temperature. You can use the same software to increase your fan speed to try and bring down that 75C. Some GPUs can have problems at higher temperatures and that is a little higher than mine was due to the overclock. IIRC I kept mine around 70C. Mine is not in a computer at the moment so I can't check it.
robintrade
Posts: 4
Joined: Tue Jun 21, 2011 2:13 pm

Re: Has my GTX 560 Ti failed

Post by robintrade »

Well Bollix47, thanks for all your help. I lowered the mem clock to 2000, but still had problems.
I then lowered the core clock to 882. No crashes since then. Its been about 48 hours.
Temp has dropped a bit, averaging around 74-75 C.
This can be marked "solved"
Post Reply