Page 1 of 1

cudaMalloc CUDAStream::Allocate failed

PostPosted: Fri Jul 23, 2010 6:57 pm
by dimilunatic
Code: Select all
--- Opening Log file [July 23 18:51:08 UTC]


# Windows GPU Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.23

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: Z:\home\dgkontopoulos\folding\GPU
Executable: Z:\home\dgkontopoulos\folding\GPU\Folding@home-Win32-GPU.exe
Arguments: -verbosity 9 -forcegpu nvidia_g80 -verbosity 9

[18:51:08] - Ask before connecting: No
[18:51:08] - User name: dimilunatic (Team 160082)
[18:51:08] - User ID: 615EB63E65848131
[18:51:08] - Machine ID: 8
[18:51:08]
[18:51:08] Loaded queue successfully.
[18:51:08] - Preparing to get new work unit...
[18:51:08] - Autosending finished units... [July 23 18:51:08 UTC]
[18:51:08] + Attempting to get work packet
[18:51:08] Trying to send all finished work units
[18:51:08] + No unsent completed units remaining.
[18:51:08] - Autosend completed
[18:51:08] - Will indicate memory of 1745 MB
[18:51:08] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 5, Stepping: 2
[18:51:08] - Connecting to assignment server
[18:51:08] Connecting to http://assign-GPU.stanford.edu:8080/
[18:51:09] Posted data.
[18:51:09] Initial: 40AB; - Successful: assigned to (171.64.65.61).
[18:51:09] + News From Folding@Home: Welcome to Folding@Home
[18:51:09] Loaded queue successfully.
[18:51:09] Connecting to http://171.64.65.61:8080/
[18:51:10] Posted data.
[18:51:10] Initial: 0000; - Receiving payload (expected size: 74390)
[18:51:15] - Downloaded at ~14 kB/s
[18:51:15] - Averaged speed for that direction ~43 kB/s
[18:51:15] + Received work.
[18:51:15] + Closed connections
[18:51:15]
[18:51:15] + Processing work unit
[18:51:15] Core required: FahCore_11.exe
[18:51:15] Core found.
[18:51:15] Working on queue slot 09 [July 23 18:51:15 UTC]
[18:51:15] + Working ...
[18:51:15] - Calling '.\FahCore_11.exe -dir work/ -suffix 09 -priority 96 -checkpoint 30 -verbose -lifeline 42 -version 623'

[18:51:15]
[18:51:15] *------------------------------*
[18:51:15] Folding@Home GPU Core
[18:51:15] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[18:51:15]
[18:51:15] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86
[18:51:15] Build host: amoeba
[18:51:15] Board Type: Nvidia
[18:51:15] Core      :
[18:51:15] Preparing to commence simulation
[18:51:15] - Looking at optimizations...
[18:51:15] DeleteFrameFiles: successfully deleted file=work/wudata_09.ckp
[18:51:15] - Created dyn
[18:51:15] - Files status OK
[18:51:15] - Expanded 73878 -> 383588 (decompressed 519.2 percent)
[18:51:15] Called DecompressByteArray: compressed_data_size=73878 data_size=383588, decompressed_data_size=383588 diff=0
[18:51:15] - Digital signature verified
[18:51:15]
[18:51:15] Project: 6606 (Run 8, Clone 125, Gen 241)
[18:51:15]
[18:51:15] Assembly optimizations on if available.
[18:51:15] Entering M.D.
Reading file work/wudata_09.tpr, VERSION 3.1.4 (single precision)
Reading file work/wudata_09.tpr, VERSION 3.1.4 (single precision)
Reading sasa-enabled ir 0 0
[18:51:21] Tpr hash work/wudata_09.tpr:  1296645041 558172639 293069673 2309319592 925558028
[18:51:21]
[18:51:21] Calling fah_main args: 14 usage=100
[18:51:21]
Initializing Nvidia gpu library
cudaMalloc CUDAStream::Allocate failed no CUDA-capable device is available
[18:51:22] mdrun_gpu returned
[18:51:22] Going to send back what have done -- stepsTotalG=0
[18:51:22] Work fraction=0.0000 steps=0.
[18:51:26] logfile size=4944 infoLength=4944 edr=0 trr=25
[18:51:26] + Opened results file
[18:51:26] - Writing 5482 bytes of core data to disk...
[18:51:26] Done: 4970 -> 1857 (compressed to 37.3 percent)
[18:51:26]   ... Done.
[18:51:26] DeleteFrameFiles: successfully deleted file=work/wudata_09.ckp
[18:51:26]
[18:51:26] Folding@home Core Shutdown: UNSTABLE_MACHINE


I'm completely clueless. This client was running perfectly until 8 hours ago, when I had to turn off the pc and get a train. Now that I arrived home, I receive this sort of error. What could have possibly gone wrong?

Re: cudaMalloc CUDAStream::Allocate failed

PostPosted: Fri Jul 23, 2010 7:22 pm
by Hyperlife
Are you running X or headless (no desktop)? What do you see when you enter this command:

Code: Select all
ls /dev/nv*

Re: cudaMalloc CUDAStream::Allocate failed

PostPosted: Fri Jul 23, 2010 8:02 pm
by dimilunatic
Hyperlife wrote:Are you running X or headless (no desktop)? What do you see when you enter this command:

Code: Select all
ls /dev/nv*

No, I'm running a common install of Ubuntu in GNOME DE on a laptop.

Code: Select all
/dev/nvidia0  /dev/nvidiactl


Should I try reinstalling the driver?

Re: cudaMalloc CUDAStream::Allocate failed

PostPosted: Fri Jul 23, 2010 10:01 pm
by Hyperlife
dimilunatic wrote:Should I try reinstalling the driver?

It shouldn't hurt, but I'd be surprised if that solved the problem. What driver version are you using?

Also, could you check to see if your wrapper is properly linked? What's the output of:

Code: Select all
ldd ~/.wine/drive_c/windows/system32/cudart.dll

Re: cudaMalloc CUDAStream::Allocate failed

PostPosted: Sat Jul 24, 2010 8:11 am
by dimilunatic
Hyperlife wrote:
dimilunatic wrote:Should I try reinstalling the driver?

It shouldn't hurt, but I'd be surprised if that solved the problem. What driver version are you using?

Also, could you check to see if your wrapper is properly linked? What's the output of:

Code: Select all
ldd ~/.wine/drive_c/windows/system32/cudart.dll

Code: Select all
linux-gate.so.1 =>  (0xf772a000)
   libcudart.so.2 => /usr/local/cuda/lib/libcudart.so.2 (0xf76b2000)
   libwine.so.1 => /usr/lib32/libwine.so.1 (0xf7572000)
   libm.so.6 => /lib32/libm.so.6 (0xf754b000)
   libc.so.6 => /lib32/libc.so.6 (0xf73f1000)
   libdl.so.2 => /lib32/libdl.so.2 (0xf73ed000)
   libpthread.so.0 => /lib32/libpthread.so.0 (0xf73d4000)
   librt.so.1 => /lib32/librt.so.1 (0xf73cb000)
   libstdc++.so.6 => /usr/lib32/libstdc++.so.6 (0xf72d5000)
   libgcc_s.so.1 => /usr/lib32/libgcc_s.so.1 (0xf72b5000)
   /lib/ld-linux.so.2 (0xf772b000)

Re: cudaMalloc CUDAStream::Allocate failed

PostPosted: Sat Jul 24, 2010 3:13 pm
by codysluder
Is this a tiny system? How much RAM does it have?

Re: cudaMalloc CUDAStream::Allocate failed

PostPosted: Sat Jul 24, 2010 3:29 pm
by dimilunatic
codysluder wrote:Is this a tiny system? How much RAM does it have?

I wouldn't call it tiny, exactly. :P It's got 4 cores and 6 gbs of ram.

Re: cudaMalloc CUDAStream::Allocate failed

PostPosted: Sun Jul 25, 2010 5:02 am
by Hyperlife
Which driver version and CUDA version are you running?

Re: cudaMalloc CUDAStream::Allocate failed

PostPosted: Sun Jul 25, 2010 7:02 am
by dimilunatic
I reinstalled the toolkit, based on this guide, like I did in the first place and now it seems to work. :e?:

Cuda Toolkit version is 2.3 and driver is 190.53 I think.