Linux CPU folding was working fine, now has halted

Moderators: Site Moderators, FAHC Science Team

Linux CPU folding was working fine, now has halted

Postby at165db » Fri Oct 12, 2012 2:19 pm

For a few days things were churning along nicely on my pc. Today I noticed that my CPU load was 0% not 800%. I'm not sure how to get things churning along again.
The key error seems to be "Examination of work files indicates 8 consecutive improper terminations of core."

Code: Select all
*********************** Log Started 2012-10-12T13:23:22Z ***********************
13:23:22:************************* Folding@home Client *************************
13:23:22:    Website: http://folding.stanford.edu/
13:23:22:  Copyright: (c) 2009-2012 Stanford University
13:23:22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
13:23:22:       Args: --child --lifeline 3440 /etc/fahclient/config.xml --run-as
13:23:22:             fahclient --pid-file=/var/run/fahclient.pid --daemon
13:23:22:     Config: /etc/fahclient/config.xml
13:23:22:******************************** Build ********************************
13:23:22:    Version: 7.1.52
13:23:22:       Date: Mar 20 2012
13:23:22:       Time: 13:19:11
13:23:22:    SVN Rev: 3515
13:23:22:     Branch: fah/trunk/client
13:23:22:   Compiler: GNU 4.6.2
13:23:22:    Options: -std=gnu++98 -O3 -funroll-loops -mfpmath=sse -ffast-math
13:23:22:             -fno-unsafe-math-optimizations -msse2
13:23:22:   Platform: linux2 3.2.0-1-amd64
13:23:22:       Bits: 64
13:23:22:       Mode: Release
13:23:22:******************************* System ********************************
13:23:22:        CPU: Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
13:23:22:     CPU ID: GenuineIntel Family 6 Model 30 Stepping 5
13:23:22:       CPUs: 8
13:23:22:     Memory: 7.79GiB
13:23:22:Free Memory: 5.90GiB
13:23:22:    Threads: POSIX_THREADS
13:23:22: On Battery: false
13:23:22: UTC offset: -4
13:23:22:        PID: 3446
13:23:22:        CWD: /var/lib/fahclient
13:23:22:         OS: Linux 3.2.0-31-generic x86_64
13:23:22:    OS Arch: AMD64
13:23:22:       GPUs: 1
13:23:22:      GPU 0: NVIDIA:1 G96 [GeForce 9500 GT]
13:23:22:       CUDA: 1.1
13:23:22:CUDA Driver: 4020
13:23:22:***********************************************************************
13:23:22:<config>
13:23:22:  <!-- Folding Slot Configuration -->
13:23:22:  <gpu v='true'/>
13:23:22:
13:23:22:  <!-- User Information -->
13:23:22:  <passkey v='********************************'/>
13:23:22:  <team v='1115'/>
13:23:22:  <user v='at165dB'/>
13:23:22:
13:23:22:  <!-- Folding Slots -->
13:23:22:</config>
13:23:22:Switching to user fahclient
13:23:22:Trying to access database...
13:23:22:Successfully acquired database lock
13:23:22:Enabled folding slot 00: READY gpu:0:"G96 [GeForce 9500 GT]"
13:23:22:Enabled folding slot 01: READY smp:8
13:23:22:WARNING:WU01:No longer matches Slot 0's configuration, migrating to FS01
13:23:22:WU01:FS01:Starting
13:23:22:WU01:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/www.stanford.edu/~pande/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 01 -suffix 01 -version 701 -lifeline 3446 -checkpoint 15 -np 8
13:23:22:WU01:FS01:Started FahCore on PID 3455
13:23:22:WU01:FS01:Core PID:3459
13:23:22:WU01:FS01:FahCore 0xa4 started
13:23:22:WU00:FS00:Connecting to assign-GPU.stanford.edu:80
13:23:23:WU01:FS01:0xa4:
13:23:23:WU01:FS01:0xa4:*------------------------------*
13:23:23:WU01:FS01:0xa4:Folding@Home Gromacs GB Core
13:23:23:WU01:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
13:23:23:WU01:FS01:0xa4:
13:23:23:WU01:FS01:0xa4:Preparing to commence simulation
13:23:23:WU01:FS01:0xa4:- Ensuring status. Please wait.
13:23:32:WU01:FS01:0xa4:- Looking at optimizations...
13:23:32:WU01:FS01:0xa4:- Working with standard loops on this execution.
13:23:32:WU01:FS01:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
13:23:32:WU01:FS01:0xa4:- Expanded 29854 -> 644556 (decompressed 2159.0 percent)
13:23:32:WU01:FS01:0xa4:Called DecompressByteArray: compressed_data_size=29854 data_size=644556, decompressed_data_size=644556 diff=0
13:23:32:WU01:FS01:0xa4:- Digital signature verified
13:23:32:WU01:FS01:0xa4:
13:23:32:WU01:FS01:0xa4:Project: 7611 (Run 0, Clone 23, Gen 263)
13:23:32:WU01:FS01:0xa4:
13:23:32:WU01:FS01:0xa4:Entering M.D.
13:23:33:Server connection id=1 on 0.0.0.0:36330 from 127.0.0.1
13:23:33:WU00:FS00:News: Welcome to Folding@Home
13:23:33:WARNING:WU00:FS00:Failed to get assignment from 'assign-GPU.stanford.edu:80': Empty work server assignment
13:23:33:WU00:FS00:Connecting to assign-GPU.stanford.edu:8080
13:23:33:WU00:FS00:News: Welcome to Folding@Home
13:23:33:WARNING:WU00:FS00:Failed to get assignment from 'assign-GPU.stanford.edu:8080': Empty work server assignment
13:23:33:ERROR:WU00:FS00:Exception: Could not get an assignment
13:23:33:WU00:FS00:Connecting to assign-GPU.stanford.edu:80
13:23:39:WU01:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
13:23:39:WU01:FS01:Starting
13:23:39:WU01:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/www.stanford.edu/~pande/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 01 -suffix 01 -version 701 -lifeline 3446 -checkpoint 15 -np 8
13:23:39:WU01:FS01:Started FahCore on PID 3480
13:23:39:WU01:FS01:Core PID:3484
13:23:39:WU01:FS01:FahCore 0xa4 started
13:23:39:WU01:FS01:0xa4:
13:23:39:WU01:FS01:0xa4:*------------------------------*
13:23:39:WU01:FS01:0xa4:Folding@Home Gromacs GB Core
13:23:39:WU01:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
13:23:39:WU01:FS01:0xa4:
13:23:39:WU01:FS01:0xa4:Preparing to commence simulation
13:23:39:WU01:FS01:0xa4:- Ensuring status. Please wait.
13:23:43:WU00:FS00:News: Welcome to Folding@Home
13:23:43:WARNING:WU00:FS00:Failed to get assignment from 'assign-GPU.stanford.edu:80': Empty work server assignment
13:23:43:WU00:FS00:Connecting to assign-GPU.stanford.edu:8080
13:23:43:WU00:FS00:News: Welcome to Folding@Home
13:23:43:WARNING:WU00:FS00:Failed to get assignment from 'assign-GPU.stanford.edu:8080': Empty work server assignment
13:23:43:ERROR:WU00:FS00:Exception: Could not get an assignment
13:23:48:WU01:FS01:0xa4:- Looking at optimizations...
13:23:48:WU01:FS01:0xa4:- Working with standard loops on this execution.
13:23:48:WU01:FS01:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
13:23:49:WU01:FS01:0xa4:- Expanded 29854 -> 644556 (decompressed 2159.0 percent)
13:23:49:WU01:FS01:0xa4:Called DecompressByteArray: compressed_data_size=29854 data_size=644556, decompressed_data_size=644556 diff=0
13:23:49:WU01:FS01:0xa4:- Digital signature verified
13:23:49:WU01:FS01:0xa4:
13:23:49:WU01:FS01:0xa4:Project: 7611 (Run 0, Clone 23, Gen 263)
13:23:49:WU01:FS01:0xa4:
13:23:49:WU01:FS01:0xa4:Entering M.D.
13:23:55:WU01:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
13:24:33:WU00:FS00:Connecting to assign-GPU.stanford.edu:80
13:24:39:WU01:FS01:Starting
13:24:39:WU01:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/www.stanford.edu/~pande/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 01 -suffix 01 -version 701 -lifeline 3446 -checkpoint 15 -np 8
13:24:39:WU01:FS01:Started FahCore on PID 3496
13:24:39:WU01:FS01:Core PID:3500
13:24:39:WU01:FS01:FahCore 0xa4 started
13:24:40:WU01:FS01:0xa4:
13:24:40:WU01:FS01:0xa4:*------------------------------*
13:24:40:WU01:FS01:0xa4:Folding@Home Gromacs GB Core
13:24:40:WU01:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
13:24:40:WU01:FS01:0xa4:
13:24:40:WU01:FS01:0xa4:Preparing to commence simulation
13:24:40:WU01:FS01:0xa4:- Ensuring status. Please wait.
13:24:44:WU00:FS00:News: Welcome to Folding@Home
13:24:44:WARNING:WU00:FS00:Failed to get assignment from 'assign-GPU.stanford.edu:80': Empty work server assignment
13:24:44:WU00:FS00:Connecting to assign-GPU.stanford.edu:8080
13:24:44:WU00:FS00:News: Welcome to Folding@Home
13:24:44:WARNING:WU00:FS00:Failed to get assignment from 'assign-GPU.stanford.edu:8080': Empty work server assignment
13:24:44:ERROR:WU00:FS00:Exception: Could not get an assignment
13:24:49:WU01:FS01:0xa4:- Looking at optimizations...
13:24:49:WU01:FS01:0xa4:- Working with standard loops on this execution.
13:24:49:WU01:FS01:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
13:24:49:WU01:FS01:0xa4:- Expanded 29854 -> 644556 (decompressed 2159.0 percent)
13:24:49:WU01:FS01:0xa4:Called DecompressByteArray: compressed_data_size=29854 data_size=644556, decompressed_data_size=644556 diff=0
13:24:49:WU01:FS01:0xa4:- Digital signature verified
13:24:49:WU01:FS01:0xa4:
13:24:49:WU01:FS01:0xa4:Project: 7611 (Run 0, Clone 23, Gen 263)
13:24:49:WU01:FS01:0xa4:
13:24:49:WU01:FS01:0xa4:Entering M.D.
13:24:55:WU01:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
at165db
 
Posts: 5
Joined: Thu Oct 11, 2012 3:52 pm

Re: Linux client, CPU folding was working fine, now has haul

Postby bollix47 » Fri Oct 12, 2012 3:10 pm

Welcome to the folding support forum at165db.

Now that you've removed your GPU slot is this still a problem?
bollix47
 
Posts: 2871
Joined: Sun Dec 02, 2007 6:04 am
Location: Canada

Re: Linux client, CPU folding was working fine, now has haul

Postby at165db » Fri Oct 12, 2012 3:47 pm

Nope :-<

Code: Select all
*********************** Log Started 2012-10-12T14:56:02Z ***********************
14:56:02:************************* Folding@home Client *************************
14:56:02:    Website: http://folding.stanford.edu/
14:56:02:  Copyright: (c) 2009-2012 Stanford University
14:56:02:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
14:56:02:       Args: --child --lifeline 3763 /etc/fahclient/config.xml --run-as
14:56:02:             fahclient --pid-file=/var/run/fahclient.pid --daemon
14:56:02:     Config: /etc/fahclient/config.xml
14:56:02:******************************** Build ********************************
14:56:02:    Version: 7.1.52
14:56:02:       Date: Mar 20 2012
14:56:02:       Time: 13:19:11
14:56:02:    SVN Rev: 3515
14:56:02:     Branch: fah/trunk/client
14:56:02:   Compiler: GNU 4.6.2
14:56:02:    Options: -std=gnu++98 -O3 -funroll-loops -mfpmath=sse -ffast-math
14:56:02:             -fno-unsafe-math-optimizations -msse2
14:56:02:   Platform: linux2 3.2.0-1-amd64
14:56:02:       Bits: 64
14:56:02:       Mode: Release
14:56:02:******************************* System ********************************
14:56:02:        CPU: Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
14:56:02:     CPU ID: GenuineIntel Family 6 Model 30 Stepping 5
14:56:02:       CPUs: 8
14:56:02:     Memory: 7.79GiB
14:56:02:Free Memory: 5.88GiB
14:56:02:    Threads: POSIX_THREADS
14:56:02: On Battery: false
14:56:02: UTC offset: -4
14:56:02:        PID: 3770
14:56:02:        CWD: /var/lib/fahclient
14:56:02:         OS: Linux 3.2.0-31-generic x86_64
14:56:02:    OS Arch: AMD64
14:56:02:       GPUs: 1
14:56:02:      GPU 0: NVIDIA:1 G96 [GeForce 9500 GT]
14:56:02:       CUDA: 1.1
14:56:02:CUDA Driver: 4020
14:56:02:***********************************************************************
14:56:02:<config>
14:56:02:  <!-- User Information -->
14:56:02:  <passkey v='********************************'/>
14:56:02:  <team v='1115'/>
14:56:02:  <user v='at165dB'/>
14:56:02:
14:56:02:  <!-- Folding Slots -->
14:56:02:</config>
14:56:02:Switching to user fahclient
14:56:02:Trying to access database...
14:56:02:Successfully acquired database lock
14:56:02:Enabled folding slot 00: READY smp:8
14:56:02:WARNING:WU01:Slot ID 1 no longer exists, migrating to FS00
14:56:02:WU01:FS00:Starting
14:56:02:WU01:FS00:Removing old file './work/01/logfile_01-20121011-165737.txt'
14:56:02:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/www.stanford.edu/~pande/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 01 -suffix 01 -version 701 -lifeline 3770 -checkpoint 15 -np 8
14:56:02:WU01:FS00:Started FahCore on PID 3778
14:56:02:WU01:FS00:Core PID:3782
14:56:02:WU01:FS00:FahCore 0xa4 started
14:56:03:WU01:FS00:0xa4:
14:56:03:WU01:FS00:0xa4:*------------------------------*
14:56:03:WU01:FS00:0xa4:Folding@Home Gromacs GB Core
14:56:03:WU01:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
14:56:03:WU01:FS00:0xa4:
14:56:03:WU01:FS00:0xa4:Preparing to commence simulation
14:56:03:WU01:FS00:0xa4:- Ensuring status. Please wait.
14:56:07:Server connection id=1 on 0.0.0.0:36330 from 127.0.0.1
14:56:12:WU01:FS00:0xa4:- Looking at optimizations...
14:56:12:WU01:FS00:0xa4:- Working with standard loops on this execution.
14:56:12:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
14:56:12:WU01:FS00:0xa4:- Expanded 29854 -> 644556 (decompressed 2159.0 percent)
14:56:12:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=29854 data_size=644556, decompressed_data_size=644556 diff=0
14:56:12:WU01:FS00:0xa4:- Digital signature verified
14:56:12:WU01:FS00:0xa4:
14:56:12:WU01:FS00:0xa4:Project: 7611 (Run 0, Clone 23, Gen 263)
14:56:12:WU01:FS00:0xa4:
14:56:12:WU01:FS00:0xa4:Entering M.D.
14:56:18:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
14:56:19:WU01:FS00:Starting
14:56:19:WU01:FS00:Removing old file './work/01/logfile_01-20121011-170428.txt'
14:56:19:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/www.stanford.edu/~pande/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 01 -suffix 01 -version 701 -lifeline 3770 -checkpoint 15 -np 8
14:56:19:WU01:FS00:Started FahCore on PID 3803
14:56:19:WU01:FS00:Core PID:3807
14:56:19:WU01:FS00:FahCore 0xa4 started
14:56:19:WU01:FS00:0xa4:
14:56:19:WU01:FS00:0xa4:*------------------------------*
14:56:19:WU01:FS00:0xa4:Folding@Home Gromacs GB Core
14:56:19:WU01:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
14:56:19:WU01:FS00:0xa4:
14:56:19:WU01:FS00:0xa4:Preparing to commence simulation
14:56:19:WU01:FS00:0xa4:- Ensuring status. Please wait.
14:56:28:WU01:FS00:0xa4:- Looking at optimizations...
14:56:28:WU01:FS00:0xa4:- Working with standard loops on this execution.
14:56:28:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
14:56:28:WU01:FS00:0xa4:- Expanded 29854 -> 644556 (decompressed 2159.0 percent)
14:56:28:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=29854 data_size=644556, decompressed_data_size=644556 diff=0
14:56:28:WU01:FS00:0xa4:- Digital signature verified
14:56:28:WU01:FS00:0xa4:
14:56:28:WU01:FS00:0xa4:Project: 7611 (Run 0, Clone 23, Gen 263)
14:56:28:WU01:FS00:0xa4:
14:56:28:WU01:FS00:0xa4:Entering M.D.
14:56:35:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
at165db
 
Posts: 5
Joined: Thu Oct 11, 2012 3:52 pm

Re: Linux client, CPU folding was working fine, now has haul

Postby at165db » Fri Oct 12, 2012 3:54 pm

Ah ha! I found where the work is stored (it sure has moved since I last used to fold years ago). /var/lib/fahclient

I stopped the client
Code: Select all
sudo /etc/init.d/FAHClient stop

I removed the contents of work and config folders under /var/lib/fahclient
Code: Select all
sudo rm -rf  /var/lib/fahclient/work/
sudo rm  /var/lib/configs/config-20121009-202802.xml

Restarted the client,
Code: Select all
sudo /etc/init.d/FAHClient start

and it now seems to be working again.
at165db
 
Posts: 5
Joined: Thu Oct 11, 2012 3:52 pm

Linux CPU folding was working fine, now has halted

Postby bollix47 » Fri Oct 12, 2012 4:19 pm

Great!

I did a check on Project: 7611 (Run 0, Clone 23, Gen 263) and there was a report for one other folder who got 0 points for it so I'm going to do a followup check in case it is a bad WU.
bollix47
 
Posts: 2871
Joined: Sun Dec 02, 2007 6:04 am
Location: Canada

Re: Linux CPU folding was working fine, now has halted

Postby bollix47 » Mon Oct 15, 2012 12:32 pm

The work unit was completed successfully by another folder.

Hi xxxxxx (team xxxxx),
Your WU (P7611 R0 C23 G263) was added to the stats database on 2012-10-14 10:11:25 for 3409.56 points of credit.
bollix47
 
Posts: 2871
Joined: Sun Dec 02, 2007 6:04 am
Location: Canada

Re: Linux CPU folding was working fine, now has halted

Postby tjlane » Mon Oct 15, 2012 5:07 pm

Hi All,

Please let me know if problems with this WU persist and I'll track down what's wrong.

Thanks,

TJ
tjlane
Pande Group Member
 
Posts: 161
Joined: Thu Jun 02, 2011 12:19 am
Location: Stanford, CA


Return to V7.1.52 Windows/Linux

Who is online

Users browsing this forum: No registered users and 1 guest

cron