2619 segfaults

Moderators: Site Moderators, FAHC Science Team

mibo

2619 segfaults

Post by mibo »

My machine is folding-stable for several weeks now, but the new 2619 WUs segfault after some % of calculation. Is 2619 more stress for my computer (and my computer is not stable?) or why do these WUs don't run smoothly?

System:
Core2Duo@3,2GHz
2x2GB RAM
Suse Linux 10.3 64Bit

Code: Select all

michael@musashi:~/fah_smp> ./fah6 -smp

Note: Please read the license agreement (fah6 -license). Further
use of this software requires that you have read and accepted this agreement.

2 cores detected


--- Opening Log file [March 31 16:34:08]


# SMP Client ##################################################################
###############################################################################

                       Folding@Home Client Version 6.01beta2

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: /home/michael/fah_smp
Executable: ./fah6
Arguments: -smp

[16:34:08] - Ask before connecting: No
[16:34:08] - User name: thorsam (Team 34361)
[16:34:08] - User ID: 5620CFA2642182AB
[16:34:08] - Machine ID: 1
[16:34:08]
[16:34:09] Loaded queue successfully.
[16:34:09]
[16:34:09] + Processing work unit
[16:34:09] Core required: FahCore_a2.exe
[16:34:09] Core found.
[16:34:09] Working on Unit 06 [March 31 16:34:09]
[16:34:09] + Working ...
[16:34:09]
[16:34:09] *------------------------------*
[16:34:09] Folding@Home Gromacs SMP Core
[16:34:09] Version 1.91 (2007)
[16:34:09]
[16:34:09] Preparing to commence simulation
[16:34:09] - Ensuring status. Please wait.
[16:34:26] - Looking at optimizations...
[16:34:26] - Working with standard loops on this execution.
[16:34:26] - Previous termination of core was improper.
[16:34:26] - Going to use standard loops.
[16:34:26] - Files status OK
[16:34:26] Error: Work unit read from disk is invalid
[16:34:26] Finalizing output
[16:34:29] - Expanded 9152049 -> 48331685 (decompressed 58.8 percent)
[16:34:30]
[16:34:30] Project: 2619 (Run 7, Clone 523, Gen 6)
[16:34:30]
[16:34:30] Entering M.D.
NNODES=4, MYRANK=2, HOSTNAME=musashi
NNODES=4, MYRANK=0, HOSTNAME=musashi
NNODES=4, MYRANK=3, HOSTNAME=musashi
NNODES=4, MYRANK=1, HOSTNAME=musashi
NODEID=3 argc=14
NODEID=2 argc=14
NODEID=1 argc=14
NODEID=0 argc=14
                         :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                 :-)  VERSION 3.3.99_development_20070720  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2006, The GROMACS development team,
            check out http://www.gromacs.org for more information.

         This program is free software; you can redistribute it and/or
          modify it under the terms of the GNU General Public License
         as published by the Free Software Foundation; either version 2
             of the License, or (at your option) any later version.

                                :-)  mdrun  (-:

Reading file work/wudata_06.tpr, VERSION 3.3 (single precision)
Note: tpx file_version 40, software version 48
starting mdrun 'TETHERED VESICLES'
125000 steps,   5000.0 ps.
[16:34:39] Completed 0 out of 125000 steps  (0%)
[16:41:15] Completed 630 out of 125000 steps  (1%)
[16:54:12] Completed 1880 out of 125000 steps  (2%)
[17:07:05] Completed 3130 out of 125000 steps  (3%)
[17:19:59] Completed 4380 out of 125000 steps  (4%)
[17:33:20] Completed 5630 out of 125000 steps  (5%)
[17:46:29] Completed 6880 out of 125000 steps  (6%)
[17:59:30] Completed 8130 out of 125000 steps  (7%)
[18:12:45] Completed 9380 out of 125000 steps  (8%)
[18:25:53] Completed 10630 out of 125000 steps  (9%)
[18:38:46] Completed 11880 out of 125000 steps  (10%)
[18:51:43] Completed 13130 out of 125000 steps  (11%)
[cli_0]: aborting job:
Fatal error in MPI_Sendrecv: Error message texts are not available
[0]0:Return code = 1
[0]1:Return code = 0, signaled with Segmentation fault
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
[19:00:12] CoreStatus = 1 (1)
[19:00:12] Client-core communications error: ERROR 0x1
[19:00:12] Deleting current work unit & continuing...
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
7im
Posts: 10189
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:

Re: 2619 segfaults

Post by 7im »

p2619 is a new project, and uses a lot more memory (around 800 MB). There have been reports that this project is not as stable as more recent projects. I wouldn't worry about your system.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
toTOW
Site Moderator
Posts: 6309
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:

Re: 2619 segfaults

Post by toTOW »

No your computer is not stable ... run stability tests and act in consequence : lower your overclock, or release memory timings a bit ;)

I had similiar errors while overclocking, and I think they are often memory related.
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
mibo

Re: 2619 segfaults

Post by mibo »

My computer does F@H when I'm at work - So I switch on the PC every morning.
Yesterday when I shutdown the machine I (as usual) ctrl+c the F@H client which had 2% of a 2619 finished.
This morning I reduced my overclock to 3,0GHz and started F@H (see output).

@toTOW: I use Prime95 for stability tests and use this computer for LAN gaming, so I'm quite confident that the PC is stable. But I welcome more strict stability tests :-)

Code: Select all

michael@musashi:~/fah_smp> ./fah6 -smp

Note: Please read the license agreement (fah6 -license). Further
use of this software requires that you have read and accepted this agreement.

2 cores detected


--- Opening Log file [April 1 05:57:58]


# SMP Client ##################################################################
###############################################################################

                       Folding@Home Client Version 6.01beta2

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: /home/michael/fah_smp
Executable: ./fah6
Arguments: -smp

[05:57:58] - Ask before connecting: No
[05:57:58] - User name: thorsam (Team 34361)
[05:57:58] - User ID: 5620CFA2642182AB
[05:57:58] - Machine ID: 1
[05:57:58]
[05:57:58] Loaded queue successfully.
[05:57:58]
[05:57:58] + Processing work unit
[05:57:58] Core required: FahCore_a2.exe
[05:57:58] Core found.
[05:57:58] Working on Unit 09 [April 1 05:57:58]
[05:57:58] + Working ...
[05:57:59]
[05:57:59] *------------------------------*
[05:57:59] Folding@Home Gromacs SMP Core
[05:57:59] Version 1.91 (2007)
[05:57:59]
[05:57:59] Preparing to commence simulation
[05:57:59] - Ensuring status. Please wait- Files status OK
[05:57:59] Error: Work unit read from disk is invalid
[05:57:59] Finalizing output
[05:58:02] - Expanded 7863527 -> 48331685 (decompressed 68.4 percent)
[05:58:02]
[05:58:02] Project: 2619 (Run 2, Clone 672, Gen 0)
[05:58:02]
[05:58:02] Assembly optimizations on if available.
[05:58:02] Entering M.D.
[05:58:08] Will resume from checkpoint file
[05:58:17] (decompressed 68.4 percent)
[05:58:17]
[05:58:17] Project: 2619 (Run 2, Clone 672, Gen 0)
[05:58:17]
[05:58:17] Entering M.D.
[05:58:23] Will resume from checkpoint file
NNODES=4, MYRANK=0, HOSTNAME=musashi
NNODES=4, MYRANK=3, HOSTNAME=musashi
NNODES=4, MYRANK=1, HOSTNAME=musashi
NNODES=4, MYRANK=2, HOSTNAME=musashi
NODEID=3 argc=14
NODEID=1 argc=14
NODEID=2 argc=14
NODEID=0 argc=14
                         :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                 :-)  VERSION 3.3.99_development_20070720  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2006, The GROMACS development team,
            check out http://www.gromacs.org for more information.

         This program is free software; you can redistribute it and/or
          modify it under the terms of the GNU General Public License
         as published by the Free Software Foundation; either version 2
             of the License, or (at your option) any later version.

                                :-)  mdrun  (-:

Reading file work/wudata_09.tpr, VERSION 3.3 (single precision)
Note: tpx file_version 40, software version 48
starting mdrun 'TETHERED VESICLES'
125000 steps,   5000.0 ps.
old size= 9984 old_crc=632819
-------------------------------------------------------
Program mdrun, VERSION 3.3.99_development_20070720
Source code file: md.c, line: 831

Fatal error:
Checkpoint error on step 3140

-------------------------------------------------------

Thanx for Using GROMACS - Have a Nice Day

Error on node 0, will try to stop all the nodes
Halting parallel program mdrun on CPU 0 out of 4

gcq#0: Thanx for Using GROMACS - Have a Nice Day

[cli_0]: aborting job:
application called MPI_Abort(MPI_COMM_WORLD, -1) - process 0
[05:58:33] eps  (3%)
[05:58:33] 3140 out of 125000 steps  (3%)
[cli_1]: aborting job:
Fatal error in MPI_Wait: Error message texts are not available
[0]0:Return code = 255
[0]1:Return code = 1
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
[05:58:37] CoreStatus = FF (255)
[05:58:37] Client-core communications error: ERROR 0xff
[05:58:37] Deleting current work unit & continuing...
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]1:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: 2619 segfaults

Post by bruce »

mibo wrote:I use Prime95 for stability tests and use this computer for LAN gaming, so I'm quite confident that the PC is stable. But I welcome more strict stability tests :-)
Prime95 is a good way to tell if your computer is too unstable to run Windows by itself but it has very poor correlation with being stable when running FAH. Check our 3rd party tools section for stress cpu. You can download it from gromacs.org here: StressCpuv2.0
toTOW
Site Moderator
Posts: 6309
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:

Re: 2619 segfaults

Post by toTOW »

Don't forget to run a memtest too ;)
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
mibo

Re: 2619 segfaults

Post by mibo »

toTOW wrote:Don't forget to run a memtest too ;)
Prime95 blend works much better for RAM testing for me. I had several "stable" memtest runs when Prime95 blend reported errors within minutes.
toTOW
Site Moderator
Posts: 6309
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:

Re: 2619 segfaults

Post by toTOW »

Prime95 is useless as it doesn't stresses the right part of the CPU ... use our tools and you might see a difference.

And remember that p2619 stresses the RAM more than the other tools does.
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
7im
Posts: 10189
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:

Re: 2619 segfaults

Post by 7im »

I've seen where memtest didn't find a memory error that the Microsoft Memory Tester did find. Always use the best tools for the job.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
Ivoshiee
Site Moderator
Posts: 822
Joined: Sun Dec 02, 2007 12:05 am
Location: Estonia

Re: 2619 segfaults

Post by Ivoshiee »

7im wrote:I've seen where memtest didn't find a memory error that the Microsoft Memory Tester did find. Always use the best tools for the job.
Just use multiple tools.
7im
Posts: 10189
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:

Re: 2619 segfaults

Post by 7im »

That's true also. Always get a second opinion with any diagnosis. Depending too heavily on only one tool can lead to problems. ;)
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
jaymode
Posts: 4
Joined: Fri Apr 04, 2008 12:53 am

Re: 2619 segfaults

Post by jaymode »

I have the same issue. The work unit crashes almost immediately for me. I have used the system stress tester and memtest to no avail. It only wants to do this work unit. Is there a way to get around this?

Code: Select all

[22:00:00] Client-core communications error: ERROR 0x1
[22:00:00] Deleting current work unit & continuing...
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]1:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[22:04:22] - Preparing to get new work unit...
[22:04:22] + Attempting to get work packet
[22:04:22] - Connecting to assignment server
[22:04:23] - Successful: assigned to (171.64.65.56).
[22:04:23] + News From Folding@Home: Welcome to Folding@Home
[22:04:23] Loaded queue successfully.
[22:04:45] + Closed connections
[22:04:50]
[22:04:50] + Processing work unit
[22:04:50] Core required: FahCore_a2.exe
[22:04:50] Core found.
[22:04:51] Working on Unit 00 [April 3 22:04:51]
[22:04:51] + Working ...
[22:04:51]
[22:04:51] *------------------------------*
[22:04:51] Folding@Home Gromacs SMP Core
[22:04:51] Version 1.91 (2007)
[22:04:51]
[22:04:51] Preparing to commence simulation
[22:04:51] - Ensuring status. Please wait.
[22:05:08] - Looking at optimizations...
[22:05:08] - Working with standard loops on this execution.
[22:05:08] - Previous termination of core was improper.
[22:05:08] - Going to use standard loops.
[22:05:08] - Files status OK
[22:05:08] Error: Work unit read from disk is invalid
[22:05:08] Finalizing output
[22:05:10] oject: 2619 (Run 4, Clone 785, Gen 0)
[22:05:10]
[22:05:10] Entering M.D.
[22:05:11] cent)
[22:05:11]
[22:05:11] Project: 2619 (Run 4, Clone 785, Gen 0)
[22:05:11]
[22:05:11] Entering M.D.
NNODES=4, MYRANK=0, HOSTNAME=JayDesktop
NNODES=4, MYRANK=3, HOSTNAME=JayDesktop
NNODES=4, MYRANK=1, HOSTNAME=JayDesktop
NNODES=4, MYRANK=2, HOSTNAME=JayDesktop
NODEID=1 argc=14
NODEID=3 argc=14
NODEID=2 argc=14
NODEID=0 argc=14
                         :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                 :-)  VERSION 3.3.99_development_20070720  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2006, The GROMACS development team,
            check out http://www.gromacs.org for more information.

         This program is free software; you can redistribute it and/or
          modify it under the terms of the GNU General Public License
         as published by the Free Software Foundation; either version 2
             of the License, or (at your option) any later version.

                                :-)  mdrun  (-:

Reading file work/wudata_00.tpr, VERSION 3.3 (single precision)
Note: tpx file_version 40, software version 48
starting mdrun 'TETHERED VESICLES'
125000 steps,   5000.0 ps.
[22:05:31] s  (0%)
[22:05:31] d 10 out of 125000 steps  (0%)
[0]0:Return code = 0, signaled with Aborted
[0]1:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
[22:05:46] CoreStatus = 0 (0)
[22:05:46] Client-core communications error: ERROR 0x0
[22:05:46] Too many errors during run. Purging queue.
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]1:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
After shutdown
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]1:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
After shutdown
After shutdown
After shutdown
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
After shutdown
After shutdown
After shutdown
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
After shutdown
After shutdown
After shutdown
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]1:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
After shutdown
After shutdown
After shutdown
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]1:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
After shutdown
After shutdown
After shutdown
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]1:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
After shutdown
After shutdown
After shutdown
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
After shutdown
After shutdown
After shutdown
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
After shutdown
After shutdown
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]1:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
[22:49:19]
[22:49:19] + Processing work unit
[22:49:19] Core required: FahCore_a2.exe
[22:49:19] Core found.
[22:49:19] Working on Unit 00 [April 3 22:49:19]
[22:49:19] + Working ...
[22:49:19]
[22:49:19] *------------------------------*
[22:49:19] Folding@Home Gromacs SMP Core
[22:49:19] Version 1.91 (2007)
[22:49:19]
[22:49:19] Preparing to commence simulation
[22:49:19] - Ensuring status. Please wait.
[22:49:36] - Looking at optimizations...
[22:49:36] - Working with standard loops on this execution.
[22:49:36] - Previous termination of core was improper.
[22:49:36] - Going to use standard loops.
[22:49:36] - Files status OK
[22:49:36] Finalizing output
[22:49:36]
[22:49:36] Project: 0 (Run 0, Clone 0, Gen 0)
[22:49:36]
[22:49:36] Error: Could not write local file.  Exiting.
[22:49:36] - Shutting down core
After shutdown
After shutdown
After shutdown
After shutdown
[22:51:36] Finalizing output
[0]0:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
[22:51:40] CoreStatus = 0 (0)
[22:51:40] Client-core communications error: ERROR 0x0
[22:51:40] - Attempting to download new core...
[22:51:40] + Downloading new core: FahCore_a2.exe
[22:51:41] + 10240 bytes downloaded
[22:51:41] + 20480 bytes downloaded
[22:51:41] + 30720 bytes downloaded
[22:51:41] + 40960 bytes downloaded
[22:51:41] + 51200 bytes downloaded
[22:51:41] + 61440 bytes downloaded
[22:51:41] + 71680 bytes downloaded
[22:51:41] + 81920 bytes downloaded
[22:51:41] + 92160 bytes downloaded
[22:51:41] + 102400 bytes downloaded
[22:51:42] + 112640 bytes downloaded
[22:51:42] + 122880 bytes downloaded
[22:51:42] + 133120 bytes downloaded
[22:51:42] + 143360 bytes downloaded
[22:51:42] + 153600 bytes downloaded
[22:51:42] + 163840 bytes downloaded
[22:51:42] + 174080 bytes downloaded
[22:51:42] + 184320 bytes downloaded
[22:51:42] + 194560 bytes downloaded
[22:51:42] + 204800 bytes downloaded
[22:51:42] + 215040 bytes downloaded
[22:51:42] + 225280 bytes downloaded
[22:51:42] + 235520 bytes downloaded
[22:51:42] + 245760 bytes downloaded
[22:51:42] + 256000 bytes downloaded
[22:51:42] + 266240 bytes downloaded
[22:51:42] + 276480 bytes downloaded
[22:51:42] + 286720 bytes downloaded
[22:51:42] + 296960 bytes downloaded
[22:51:42] + 307200 bytes downloaded
[22:51:43] + 317440 bytes downloaded
[22:51:43] + 327680 bytes downloaded
[22:51:43] + 337920 bytes downloaded
[22:51:43] + 348160 bytes downloaded
[22:51:43] + 358400 bytes downloaded
[22:51:43] + 368640 bytes downloaded
[22:51:43] + 378880 bytes downloaded
[22:51:43] + 389120 bytes downloaded
[22:51:43] + 399360 bytes downloaded
[22:51:43] + 409600 bytes downloaded
[22:51:43] + 419840 bytes downloaded
[22:51:43] + 430080 bytes downloaded
[22:51:43] + 440320 bytes downloaded
[22:51:43] + 450560 bytes downloaded
[22:51:43] + 460800 bytes downloaded
[22:51:43] + 471040 bytes downloaded
[22:51:43] + 481280 bytes downloaded
[22:51:43] + 491520 bytes downloaded
[22:51:43] + 501760 bytes downloaded
[22:51:43] + 512000 bytes downloaded
[22:51:43] + 522240 bytes downloaded
[22:51:43] + 532480 bytes downloaded
[22:51:43] + 542720 bytes downloaded
[22:51:43] + 552960 bytes downloaded
[22:51:43] + 563200 bytes downloaded
[22:51:43] + 573440 bytes downloaded
[22:51:43] + 583680 bytes downloaded
[22:51:43] + 593920 bytes downloaded
[22:51:43] + 604160 bytes downloaded
[22:51:43] + 614400 bytes downloaded
[22:51:43] + 624640 bytes downloaded
[22:51:43] + 634880 bytes downloaded
[22:51:43] + 645120 bytes downloaded
[22:51:43] + 655360 bytes downloaded
[22:51:43] + 665600 bytes downloaded
[22:51:43] + 675840 bytes downloaded
[22:51:43] + 686080 bytes downloaded
[22:51:43] + 696320 bytes downloaded
[22:51:43] + 706560 bytes downloaded
[22:51:43] + 716800 bytes downloaded
[22:51:43] + 727040 bytes downloaded
[22:51:43] + 737280 bytes downloaded
[22:51:43] + 747520 bytes downloaded
[22:51:43] + 757760 bytes downloaded
[22:51:43] + 768000 bytes downloaded
[22:51:43] + 778240 bytes downloaded
[22:51:43] + 788480 bytes downloaded
[22:51:43] + 798720 bytes downloaded
[22:51:43] + 808960 bytes downloaded
[22:51:44] + 819200 bytes downloaded
[22:51:44] + 829440 bytes downloaded
[22:51:44] + 839680 bytes downloaded
[22:51:44] + 849920 bytes downloaded
[22:51:44] + 860160 bytes downloaded
[22:51:44] + 870400 bytes downloaded
[22:51:44] + 880640 bytes downloaded
[22:51:44] + 890880 bytes downloaded
[22:51:44] + 901120 bytes downloaded
[22:51:44] + 911360 bytes downloaded
[22:51:44] + 921600 bytes downloaded
[22:51:44] + 931840 bytes downloaded
[22:51:44] + 942080 bytes downloaded
[22:51:44] + 952320 bytes downloaded
[22:51:44] + 962560 bytes downloaded
[22:51:44] + 972800 bytes downloaded
[22:51:44] + 983040 bytes downloaded
[22:51:44] + 993280 bytes downloaded
[22:51:44] + 1003520 bytes downloaded
[22:51:44] + 1013760 bytes downloaded
[22:51:44] + 1024000 bytes downloaded
[22:51:44] + 1034240 bytes downloaded
[22:51:44] + 1044480 bytes downloaded
[22:51:44] + 1054720 bytes downloaded
[22:51:44] + 1064960 bytes downloaded
[22:51:44] + 1075200 bytes downloaded
[22:51:44] + 1085440 bytes downloaded
[22:51:44] + 1095680 bytes downloaded
[22:51:44] + 1105920 bytes downloaded
[22:51:44] + 1116160 bytes downloaded
[22:51:44] + 1126400 bytes downloaded
[22:51:44] + 1136640 bytes downloaded
[22:51:44] + 1146880 bytes downloaded
[22:51:44] + 1157120 bytes downloaded
[22:51:44] + 1167360 bytes downloaded
[22:51:44] + 1177600 bytes downloaded
[22:51:44] + 1187840 bytes downloaded
[22:51:44] + 1198080 bytes downloaded
[22:51:44] + 1208320 bytes downloaded
[22:51:44] + 1218560 bytes downloaded
[22:51:44] + 1228800 bytes downloaded
[22:51:44] + 1239040 bytes downloaded
[22:51:44] + 1249280 bytes downloaded
[22:51:44] + 1259520 bytes downloaded
[22:51:44] + 1269760 bytes downloaded
[22:51:44] + 1280000 bytes downloaded
[22:51:44] + 1290240 bytes downloaded
[22:51:44] + 1300480 bytes downloaded
[22:51:44] + 1310720 bytes downloaded
[22:51:44] + 1320960 bytes downloaded
[22:51:44] + 1331200 bytes downloaded
[22:51:44] + 1341440 bytes downloaded
[22:51:44] + 1351680 bytes downloaded
[22:51:44] + 1361920 bytes downloaded
[22:51:44] + 1372160 bytes downloaded
[22:51:44] + 1382400 bytes downloaded
[22:51:44] + 1392640 bytes downloaded
[22:51:45] + 1402880 bytes downloaded
[22:51:45] + 1413120 bytes downloaded
[22:51:45] + 1423360 bytes downloaded
[22:51:45] + 1433600 bytes downloaded
[22:51:45] + 1443840 bytes downloaded
[22:51:45] + 1454080 bytes downloaded
[22:51:45] + 1464320 bytes downloaded
[22:51:45] + 1474560 bytes downloaded
[22:51:45] + 1484800 bytes downloaded
[22:51:45] + 1495040 bytes downloaded
[22:51:45] + 1505280 bytes downloaded
[22:51:45] + 1515520 bytes downloaded
[22:51:45] + 1525760 bytes downloaded
[22:51:45] + 1536000 bytes downloaded
[22:51:45] + 1546240 bytes downloaded
[22:51:45] + 1556480 bytes downloaded
[22:51:45] + 1566720 bytes downloaded
[22:51:45] + 1576960 bytes downloaded
[22:51:45] + 1587200 bytes downloaded
[22:51:45] + 1597440 bytes downloaded
[22:51:45] + 1607680 bytes downloaded
[22:51:45] + 1617920 bytes downloaded
[22:51:45] + 1628160 bytes downloaded
[22:51:45] + 1638400 bytes downloaded
[22:51:45] + 1648640 bytes downloaded
[22:51:45] + 1658880 bytes downloaded
[22:51:45] + 1669120 bytes downloaded
[22:51:45] + 1679360 bytes downloaded
[22:51:45] + 1689600 bytes downloaded
[22:51:45] + 1699840 bytes downloaded
[22:51:45] + 1710080 bytes downloaded
[22:51:45] + 1720320 bytes downloaded
[22:51:45] + 1730560 bytes downloaded
[22:51:45] + 1740800 bytes downloaded
[22:51:45] + 1751040 bytes downloaded
[22:51:45] + 1761280 bytes downloaded
[22:51:45] + 1771520 bytes downloaded
[22:51:45] + 1781760 bytes downloaded
[22:51:45] + 1792000 bytes downloaded
[22:51:45] + 1802240 bytes downloaded
[22:51:45] + 1810930 bytes downloaded
[22:51:45] Verifying core Core_a2.fah...
[22:51:45] Signature is VALID
[22:51:45]
[22:51:45] Trying to unzip core FahCore_a2.exe
[22:51:46] Decompressed FahCore_a2.exe (4515008 bytes) successfully
[22:51:46] + Core successfully engaged
[22:51:46] Deleting current work unit & continuing...
After shutdown
After shutdown
After shutdown
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]1:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
[22:56:07] - Preparing to get new work unit...
[22:56:07] + Attempting to get work packet
[22:56:07] - Connecting to assignment server
[22:56:08] - Successful: assigned to (171.64.65.56).
[22:56:08] + News From Folding@Home: Welcome to Folding@Home
[22:56:08] Loaded queue successfully.
[22:56:32] + Closed connections
[22:56:37]
[22:56:37] + Processing work unit
[22:56:37] Core required: FahCore_a2.exe
[22:56:37] Core found.
[22:56:37] Working on Unit 01 [April 3 22:56:37]
[22:56:37] + Working ...
[22:56:37]
[22:56:37] *------------------------------*
[22:56:37] Folding@Home Gromacs SMP Core
[22:56:37] Version 1.91 (2007)
[22:56:37]
[22:56:37] Preparing to commence simulation
[22:56:37] - Ensuring status. Please wait.
[22:56:54] - Looking at optimizations...
[22:56:54] - Working with standard loops on this execution.
[22:56:54] - Previous termination of core was improper.
[22:56:54] - Going to use standard loops.
[22:56:54] - Files status OK
[22:56:54] Error: Work unit read from disk is invalid
[22:56:54] Finalizing output
[22:56:57] (decompressed 68.4 percent)
[22:56:58] 85 (decompressed 68.4 percent)
[22:56:58]
[22:56:58] Project: 2619 (Run 4, Clone 785, Gen 0)
[22:56:58]
[22:56:58] Entering M.D.
NNODES=4, MYRANK=1, HOSTNAME=JayDesktop
NNODES=4, MYRANK=2, HOSTNAME=JayDesktop
NNODES=4, MYRANK=0, HOSTNAME=JayDesktop
NNODES=4, MYRANK=3, HOSTNAME=JayDesktop
NODEID=1 argc=14
NODEID=3 argc=14
NODEID=2 argc=14
NODEID=0 argc=14
                         :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                 :-)  VERSION 3.3.99_development_20070720  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2006, The GROMACS development team,
            check out http://www.gromacs.org for more information.

         This program is free software; you can redistribute it and/or
          modify it under the terms of the GNU General Public License
         as published by the Free Software Foundation; either version 2
             of the License, or (at your option) any later version.

                                :-)  mdrun  (-:

Reading file work/wudata_01.tpr, VERSION 3.3 (single precision)
Note: tpx file_version 40, software version 48
starting mdrun 'TETHERED VESICLES'
125000 steps,   5000.0 ps.
[22:57:18] s  (0%)
[22:57:18] d 10 out of 125000 steps  (0%)
[cli_1]: aborting job:
Fatal error in MPI_Wait: Error message texts are not available
[0]0:Return code = 0, signaled with Aborted
[0]1:Return code = 1
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
[22:57:34] CoreStatus = 1 (1)
[22:57:34] Client-core communications error: ERROR 0x1
[22:57:34] Deleting current work unit & continuing...
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]1:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[23:01:56] - Preparing to get new work unit...
[23:01:56] + Attempting to get work packet
[23:01:56] - Connecting to assignment server
[23:01:56] - Successful: assigned to (171.64.65.56).
[23:01:56] + News From Folding@Home: Welcome to Folding@Home
[23:01:56] Loaded queue successfully.
[23:02:24] + Closed connections
[23:02:29]
[23:02:29] + Processing work unit
[23:02:29] Core required: FahCore_a2.exe
[23:02:29] Core found.
[23:02:29] Working on Unit 02 [April 3 23:02:29]
[23:02:29] + Working ...
[23:02:29]
[23:02:29] *------------------------------*
[23:02:29] Folding@Home Gromacs SMP Core
[23:02:29] Version 1.91 (2007)
[23:02:29]
[23:02:29] Preparing to commence simulation
[23:02:29] - Ensuring status. Please wait.
[23:02:46] - Looking at optimizations...
[23:02:46] - Working with standard loops on this execution.
[23:02:46] - Previous termination of core was improper.
[23:02:46] - Going to use standard loops.
[23:02:46] - Files status OK
[23:02:46] Error: Work unit read from disk is invalid
[23:02:46] Finalizing output
[23:02:49] - Expanded 7860224 -> 48331685 (decompressed 68.4 percent)
[23:02:50]
[23:02:50] Project: 2Entering M.D.
[23:02:50] one 785, Gen 0)
[23:02:50]
[23:02:50] Entering M.D.
NNODES=4, MYRANK=2, HOSTNAME=JayDesktop
NNODES=4, MYRANK=3, HOSTNAME=JayDesktop
NNODES=4, MYRANK=0, HOSTNAME=JayDesktop
NNODES=4, MYRANK=1, HOSTNAME=JayDesktop
NODEID=2 argc=14
NODEID=0 argc=14
NODEID=3 argc=14
NODEID=1 argc=14
                         :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                 :-)  VERSION 3.3.99_development_20070720  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2006, The GROMACS development team,
            check out http://www.gromacs.org for more information.

         This program is free software; you can redistribute it and/or
          modify it under the terms of the GNU General Public License
         as published by the Free Software Foundation; either version 2
             of the License, or (at your option) any later version.

                                :-)  mdrun  (-:

Reading file work/wudata_02.tpr, VERSION 3.3 (single precision)
Note: tpx file_version 40, software version 48
starting mdrun 'TETHERED VESICLES'
125000 steps,   5000.0 ps.
[23:03:10] Completed 10 out of 125000 steps  (0%)
[cli_1]: aborting job:
Fatal error in MPI_Wait: Error message texts are not available
[0]0:Return code = 0, signaled with Aborted
[0]1:Return code = 1
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
[23:03:25] CoreStatus = 1 (1)
[23:03:25] Client-core communications error: ERROR 0x1
[23:03:25] Deleting current work unit & continuing...
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]1:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[23:07:47] - Preparing to get new work unit...
[23:07:47] + Attempting to get work packet
[23:07:47] - Connecting to assignment server
[23:07:47] - Successful: assigned to (171.64.65.56).
[23:07:47] + News From Folding@Home: Welcome to Folding@Home
[23:07:47] Loaded queue successfully.
[23:08:11] + Closed connections
[23:08:16]
[23:08:16] + Processing work unit
[23:08:16] Core required: FahCore_a2.exe
[23:08:16] Core found.
[23:08:16] Working on Unit 03 [April 3 23:08:16]
[23:08:16] + Working ...
[23:08:16]
[23:08:16] *------------------------------*
[23:08:16] Folding@Home Gromacs SMP Core
[23:08:16] Version 1.91 (2007)
[23:08:16]
[23:08:16] Preparing to commence simulation
[23:08:16] - Ensuring status. Please wait.
[23:08:33] - Looking at optimizations...
[23:08:33] - Working with standard loops on this execution.
[23:08:33] - Previous termination of core was improper.
[23:08:33] - Going to use standard loops.
[23:08:33] - Filesng output
[23:08:33]
[23:08:33] Error: Work Finalizing output
[23:08:33] k is invalid
[23:08:33] Finalizing output
[23:08:35] - Expanded 7860224 -> 48331685 (decompressed 68.4 percent)
[23:08:36]
[23:08:36] Project: 2619 (Run 4, Clone 785, Gen 0)
[23:08:36]
[23:08:36] Entering M.D.
NNODES=4, MYRANK=1, HOSTNAME=JayDesktop
NNODES=4, MYRANK=3, HOSTNAME=JayDesktop
NNODES=4, MYRANK=2, HOSTNAME=JayDesktop
NNODES=4, MYRANK=0, HOSTNAME=JayDesktop
NODEID=1 argc=14
NODEID=3 argc=14
NODEID=2 argc=14
NODEID=0 argc=14
                         :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                 :-)  VERSION 3.3.99_development_20070720  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2006, The GROMACS development team,
            check out http://www.gromacs.org for more information.

         This program is free software; you can redistribute it and/or
          modify it under the terms of the GNU General Public License
         as published by the Free Software Foundation; either version 2
             of the License, or (at your option) any later version.

                                :-)  mdrun  (-:

Reading file work/wudata_03.tpr, VERSION 3.3 (single precision)
Note: tpx file_version 40, software version 48
starting mdrun 'TETHERED VESICLES'
125000 steps,   5000.0 ps.
[23:08:57] s  (0%)
[23:08:57] d 10 out of 125000 steps  (0%)
[0]0:Return code = 0, signaled with Aborted
[0]1:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
[23:09:12] CoreStatus = 0 (0)
[23:09:12] Client-core communications error: ERROR 0x0
[23:09:12] - Attempting to download new core...
[23:09:12] + Downloading new core: FahCore_a2.exe
[23:09:13] + 10240 bytes downloaded
[23:09:13] + 20480 bytes downloaded
[23:09:13] + 30720 bytes downloaded
[23:09:13] + 40960 bytes downloaded
[23:09:13] + 51200 bytes downloaded
[23:09:13] + 61440 bytes downloaded
[23:09:13] + 71680 bytes downloaded
[23:09:13] + 81920 bytes downloaded
[23:09:13] + 92160 bytes downloaded
[23:09:13] + 102400 bytes downloaded
[23:09:13] + 112640 bytes downloaded
[23:09:13] + 122880 bytes downloaded
[23:09:14] + 133120 bytes downloaded
[23:09:14] + 143360 bytes downloaded
[23:09:14] + 153600 bytes downloaded
[23:09:14] + 163840 bytes downloaded
[23:09:14] + 174080 bytes downloaded
[23:09:14] + 184320 bytes downloaded
[23:09:14] + 194560 bytes downloaded
[23:09:14] + 204800 bytes downloaded
[23:09:14] + 215040 bytes downloaded
[23:09:14] + 225280 bytes downloaded
[23:09:14] + 235520 bytes downloaded
[23:09:14] + 245760 bytes downloaded
[23:09:14] + 256000 bytes downloaded
[23:09:14] + 266240 bytes downloaded
[23:09:14] + 276480 bytes downloaded
[23:09:14] + 286720 bytes downloaded
[23:09:14] + 296960 bytes downloaded
[23:09:14] + 307200 bytes downloaded
[23:09:14] + 317440 bytes downloaded
[23:09:14] + 327680 bytes downloaded
[23:09:14] + 337920 bytes downloaded
[23:09:15] + 348160 bytes downloaded
[23:09:15] + 358400 bytes downloaded
[23:09:15] + 368640 bytes downloaded
[23:09:15] + 378880 bytes downloaded
[23:09:15] + 389120 bytes downloaded
[23:09:15] + 399360 bytes downloaded
[23:09:15] + 409600 bytes downloaded
[23:09:15] + 419840 bytes downloaded
[23:09:15] + 430080 bytes downloaded
[23:09:15] + 440320 bytes downloaded
[23:09:15] + 450560 bytes downloaded
[23:09:15] + 460800 bytes downloaded
[23:09:15] + 471040 bytes downloaded
[23:09:15] + 481280 bytes downloaded
[23:09:15] + 491520 bytes downloaded
[23:09:15] + 501760 bytes downloaded
[23:09:15] + 512000 bytes downloaded
[23:09:15] + 522240 bytes downloaded
[23:09:15] + 532480 bytes downloaded
[23:09:15] + 542720 bytes downloaded
[23:09:15] + 552960 bytes downloaded
[23:09:15] + 563200 bytes downloaded
[23:09:15] + 573440 bytes downloaded
[23:09:15] + 583680 bytes downloaded
[23:09:15] + 593920 bytes downloaded
[23:09:15] + 604160 bytes downloaded
[23:09:15] + 614400 bytes downloaded
[23:09:15] + 624640 bytes downloaded
[23:09:15] + 634880 bytes downloaded
[23:09:15] + 645120 bytes downloaded
[23:09:15] + 655360 bytes downloaded
[23:09:15] + 665600 bytes downloaded
[23:09:15] + 675840 bytes downloaded
[23:09:15] + 686080 bytes downloaded
[23:09:15] + 696320 bytes downloaded
[23:09:15] + 706560 bytes downloaded
[23:09:15] + 716800 bytes downloaded
[23:09:15] + 727040 bytes downloaded
[23:09:15] + 737280 bytes downloaded
[23:09:15] + 747520 bytes downloaded
[23:09:15] + 757760 bytes downloaded
[23:09:16] + 768000 bytes downloaded
[23:09:16] + 778240 bytes downloaded
[23:09:16] + 788480 bytes downloaded
[23:09:16] + 798720 bytes downloaded
[23:09:16] + 808960 bytes downloaded
[23:09:16] + 819200 bytes downloaded
[23:09:16] + 829440 bytes downloaded
[23:09:16] + 839680 bytes downloaded
[23:09:16] + 849920 bytes downloaded
[23:09:16] + 860160 bytes downloaded
[23:09:16] + 870400 bytes downloaded
[23:09:16] + 880640 bytes downloaded
[23:09:16] + 890880 bytes downloaded
[23:09:16] + 901120 bytes downloaded
[23:09:16] + 911360 bytes downloaded
[23:09:16] + 921600 bytes downloaded
[23:09:16] + 931840 bytes downloaded
[23:09:16] + 942080 bytes downloaded
[23:09:16] + 952320 bytes downloaded
[23:09:16] + 962560 bytes downloaded
[23:09:16] + 972800 bytes downloaded
[23:09:16] + 983040 bytes downloaded
[23:09:16] + 993280 bytes downloaded
[23:09:16] + 1003520 bytes downloaded
[23:09:16] + 1013760 bytes downloaded
[23:09:16] + 1024000 bytes downloaded
[23:09:16] + 1034240 bytes downloaded
[23:09:16] + 1044480 bytes downloaded
[23:09:16] + 1054720 bytes downloaded
[23:09:16] + 1064960 bytes downloaded
[23:09:16] + 1075200 bytes downloaded
[23:09:16] + 1085440 bytes downloaded
[23:09:16] + 1095680 bytes downloaded
[23:09:16] + 1105920 bytes downloaded
[23:09:16] + 1116160 bytes downloaded
[23:09:16] + 1126400 bytes downloaded
[23:09:16] + 1136640 bytes downloaded
[23:09:16] + 1146880 bytes downloaded
[23:09:16] + 1157120 bytes downloaded
[23:09:16] + 1167360 bytes downloaded
[23:09:16] + 1177600 bytes downloaded
[23:09:16] + 1187840 bytes downloaded
[23:09:16] + 1198080 bytes downloaded
[23:09:16] + 1208320 bytes downloaded
[23:09:16] + 1218560 bytes downloaded
[23:09:16] + 1228800 bytes downloaded
[23:09:16] + 1239040 bytes downloaded
[23:09:16] + 1249280 bytes downloaded
[23:09:16] + 1259520 bytes downloaded
[23:09:16] + 1269760 bytes downloaded
[23:09:16] + 1280000 bytes downloaded
[23:09:16] + 1290240 bytes downloaded
[23:09:16] + 1300480 bytes downloaded
[23:09:16] + 1310720 bytes downloaded
[23:09:16] + 1320960 bytes downloaded
[23:09:16] + 1331200 bytes downloaded
[23:09:16] + 1341440 bytes downloaded
[23:09:16] + 1351680 bytes downloaded
[23:09:16] + 1361920 bytes downloaded
[23:09:16] + 1372160 bytes downloaded
[23:09:16] + 1382400 bytes downloaded
[23:09:16] + 1392640 bytes downloaded
[23:09:16] + 1402880 bytes downloaded
[23:09:16] + 1413120 bytes downloaded
[23:09:17] + 1423360 bytes downloaded
[23:09:17] + 1433600 bytes downloaded
[23:09:17] + 1443840 bytes downloaded
[23:09:17] + 1454080 bytes downloaded
[23:09:17] + 1464320 bytes downloaded
[23:09:17] + 1474560 bytes downloaded
[23:09:17] + 1484800 bytes downloaded
[23:09:17] + 1495040 bytes downloaded
[23:09:17] + 1505280 bytes downloaded
[23:09:17] + 1515520 bytes downloaded
[23:09:17] + 1525760 bytes downloaded
[23:09:17] + 1536000 bytes downloaded
[23:09:17] + 1546240 bytes downloaded
[23:09:17] + 1556480 bytes downloaded
[23:09:17] + 1566720 bytes downloaded
[23:09:17] + 1576960 bytes downloaded
[23:09:17] + 1587200 bytes downloaded
[23:09:17] + 1597440 bytes downloaded
[23:09:17] + 1607680 bytes downloaded
[23:09:17] + 1617920 bytes downloaded
[23:09:17] + 1628160 bytes downloaded
[23:09:17] + 1638400 bytes downloaded
[23:09:17] + 1648640 bytes downloaded
[23:09:17] + 1658880 bytes downloaded
[23:09:17] + 1669120 bytes downloaded
[23:09:17] + 1679360 bytes downloaded
[23:09:17] + 1689600 bytes downloaded
[23:09:17] + 1699840 bytes downloaded
[23:09:17] + 1710080 bytes downloaded
[23:09:17] + 1720320 bytes downloaded
[23:09:17] + 1730560 bytes downloaded
[23:09:17] + 1740800 bytes downloaded
[23:09:17] + 1751040 bytes downloaded
[23:09:17] + 1761280 bytes downloaded
[23:09:17] + 1771520 bytes downloaded
[23:09:17] + 1781760 bytes downloaded
[23:09:17] + 1792000 bytes downloaded
[23:09:17] + 1802240 bytes downloaded
[23:09:17] + 1810930 bytes downloaded
[23:09:17] Verifying core Core_a2.fah...
[23:09:17] Signature is VALID
[23:09:17]
[23:09:17] Trying to unzip core FahCore_a2.exe
[23:09:18] Decompressed FahCore_a2.exe (4515008 bytes) successfully
[23:09:18] + Core successfully engaged
[23:09:18] Deleting current work unit & continuing...
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
[23:13:40] - Preparing to get new work unit...
[23:13:40] + Attempting to get work packet
[23:13:40] - Connecting to assignment server
[23:13:40] - Successful: assigned to (171.64.65.56).
[23:13:40] + News From Folding@Home: Welcome to Folding@Home
[23:13:41] Loaded queue successfully.
[23:14:06] + Closed connections
[23:14:11]
[23:14:11] + Processing work unit
[23:14:11] Core required: FahCore_a2.exe
[23:14:11] Core found.
[23:14:12] Working on Unit 04 [April 3 23:14:12]
[23:14:12] + Working ...
[23:14:12]
[23:14:12] *------------------------------*
[23:14:12] Folding@Home Gromacs SMP Core
[23:14:12] Version 1.91 (2007)
[23:14:12]
[23:14:12] Preparing to commence simulation
[23:14:12] - Ensuring status. Please wait.
[23:14:29] - Looking at optimizations...
[23:14:29] - Working with standard loops on this execution.
[23:14:29] - Previous termination of core was improper.
[23:14:29] - Going to use standard loops.
[23:14:29] - Files status OK
[23:14:29] Error: Work unit read from disk is invalid
[23:14:29] Finalizing output
[23:14:32] (decompressed 68.4 percent)
[23:14:32] 85 (decompressed 68.4 percent)
[23:14:33]
[23:14:33] Project: 2Entering M.D.
[23:14:33] one 785, Gen 0)
[23:14:33]
[23:14:33] Entering M.D.
NNODES=4, MYRANK=2, HOSTNAME=JayDesktop
NNODES=4, MYRANK=3, HOSTNAME=JayDesktop
NNODES=4, MYRANK=1, HOSTNAME=JayDesktop
NNODES=4, MYRANK=0, HOSTNAME=JayDesktop
NODEID=0 argc=14
NODEID=2 argc=14
NODEID=3 argc=14
NODEID=1 argc=14
                         :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                 :-)  VERSION 3.3.99_development_20070720  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2006, The GROMACS development team,
            check out http://www.gromacs.org for more information.

         This program is free software; you can redistribute it and/or
          modify it under the terms of the GNU General Public License
         as published by the Free Software Foundation; either version 2
             of the License, or (at your option) any later version.

                                :-)  mdrun  (-:

Reading file work/wudata_04.tpr, VERSION 3.3 (single precision)
Note: tpx file_version 40, software version 48
starting mdrun 'TETHERED VESICLES'
125000 steps,   5000.0 ps.
[23:14:52] s  (0%)
[23:14:52] d 10 out of 125000 steps  (0%)
[0]0:Return code = 0, signaled with Aborted
[0]1:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
[23:15:07] CoreStatus = 0 (0)
[23:15:07] Client-core communications error: ERROR 0x0
[23:15:07] Deleting current work unit & continuing...
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]1:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[23:19:29] - Preparing to get new work unit...
[23:19:29] + Attempting to get work packet
[23:19:29] - Connecting to assignment server
[23:19:30] - Successful: assigned to (171.64.65.56).
[23:19:30] + News From Folding@Home: Welcome to Folding@Home
[23:19:30] Loaded queue successfully.
[23:19:57] + Closed connections
[23:20:02]
[23:20:02] + Processing work unit
[23:20:02] Core required: FahCore_a2.exe
[23:20:02] Core found.
[23:20:02] Working on Unit 05 [April 3 23:20:02]
[23:20:02] + Working ...
[23:20:02]
[23:20:02] *------------------------------*
[23:20:02] Folding@Home Gromacs SMP Core
[23:20:02] Version 1.91 (2007)
[23:20:02]
[23:20:02] Preparing to commence simulation
[23:20:02] - Ensuring status. Please wait.
[23:20:19] - Looking at optimizations...
[23:20:19] - Working with standard loops on this execution.
[23:20:19] - Previous termination of core was improper.
[23:20:19] - Going to use standard loops.
[23:20:19] - Files status OK
[23:20:19] Error: Work unit read from disk is invalid
[23:20:19] Finalizing output
[23:20:23] - Expanded 7860224 -> 48331685 (decomprEntering M.D.
[23:20:23] cent)
[23:20:23]
[23:20:23] Project: 2619 (Run 4, Clone 785, Gen 0)
[23:20:23]
[23:20:23] Entering M.D.
NNODES=4, MYRANK=0, HOSTNAME=JayDesktop
NNODES=4, MYRANK=3, HOSTNAME=JayDesktop
NNODES=4, MYRANK=2, HOSTNAME=JayDesktop
NNODES=4, MYRANK=1, HOSTNAME=JayDesktop
NODEID=0 argc=14
NODEID=3 argc=14
NODEID=1 argc=14
NODEID=2 argc=14
                         :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                 :-)  VERSION 3.3.99_development_20070720  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2006, The GROMACS development team,
            check out http://www.gromacs.org for more information.

         This program is free software; you can redistribute it and/or
          modify it under the terms of the GNU General Public License
         as published by the Free Software Foundation; either version 2
             of the License, or (at your option) any later version.

                                :-)  mdrun  (-:

Reading file work/wudata_05.tpr, VERSION 3.3 (single precision)
Note: tpx file_version 40, software version 48
starting mdrun 'TETHERED VESICLES'
125000 steps,   5000.0 ps.
[23:20:43] s  (0%)
[23:20:43] d 10 out of 125000 steps  (0%)
[0]0:Return code = 0, signaled with Aborted
[0]1:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
[23:20:58] CoreStatus = 0 (0)
[23:20:58] Client-core communications error: ERROR 0x0
[23:20:58] Deleting current work unit & continuing...
After shutdown
[0]0:Return code = 0, signaled with Quit
[0]1:Return code = 0, signaled with Quit
[0]2:Return code = 0, signaled with Quit
[23:25:20] - Preparing to get new work unit...
[23:25:20] + Attempting to get work packet
[23:25:20] - Connecting to assignment server
[23:25:20] - Successful: assigned to (171.64.65.56).
[23:25:20] + News From Folding@Home: Welcome to Folding@Home
[23:25:21] Loaded queue successfully.
[23:25:45] + Closed connections
[23:25:50]
[23:25:50] + Processing work unit
[23:25:50] Core required: FahCore_a2.exe
[23:25:50] Core found.
[23:25:51] Working on Unit 06 [April 3 23:25:51]
[23:25:51] + Working ...
[23:25:51]
[23:25:51] *------------------------------*
[23:25:51] Folding@Home Gromacs SMP Core
[23:25:51] Version 1.91 (2007)
[23:25:51]
[23:25:51] Preparing to commence simulation
[23:25:51] - Ensuring status. Please wait.
[23:26:08] - Looking at optimizations...
[23:26:08] - Working with standard loops on this execution.
[23:26:08] - Previous termination of core was improper.
[23:26:08] - Going to use standard loops.
[23:26:08] - Files status OK
[23:26:08] Error: Work unit read from disk is invalid
[23:26:08] Finalizing output
[23:26:11] oject: 2619 (Run 4, Clone 785, Gen 0)
[23:26:11]
[23:26:11] Entering M.D.
[23:26:11] cent)
[23:26:12] 5, Gen 0)
[23:26:12]
[23:26:12] 619 (Run 4, Clone 785, Gen 0)
[23:26:12]
[23:26:12] Entering M.D.
NNODES=4, MYRANK=0, HOSTNAME=JayDesktop
NNODES=4, MYRANK=2, HOSTNAME=JayDesktop
NNODES=4, MYRANK=3, HOSTNAME=JayDesktop
NNODES=4, MYRANK=1, HOSTNAME=JayDesktop
NODEID=3 argc=14
NODEID=1 argc=14
NODEID=0 argc=14
NODEID=2 argc=14
                         :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                 :-)  VERSION 3.3.99_development_20070720  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2006, The GROMACS development team,
            check out http://www.gromacs.org for more information.

         This program is free software; you can redistribute it and/or
          modify it under the terms of the GNU General Public License
         as published by the Free Software Foundation; either version 2
             of the License, or (at your option) any later version.

                                :-)  mdrun  (-:

Reading file work/wudata_06.tpr, VERSION 3.3 (single precision)
Note: tpx file_version 40, software version 48
starting mdrun 'TETHERED VESICLES'
125000 steps,   5000.0 ps.
[23:26:31] Completed 10 out of 125000 steps  (0%)
[cli_1]: aborting job:
Fatal error in MPI_Wait: Error message texts are not available
[0]0:Return code = 0, signaled with Aborted
[0]1:Return code = 1
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
[23:26:46] CoreStatus = 1 (1)
[23:26:46] Client-core communications error: ERROR 0x1
[23:26:46]
Folding@Home will go to sleep for 1 day as there have been 5 consecutive Cores executed which failed to complete a work unit.
[23:26:46] (To wake it up early, quit the application and restart it.)
[23:26:46] If problems persist, please visit our website at http://folding.stanford.edu for help.
[23:26:46] + Sleeping...
7im
Posts: 10189
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:

Re: 2619 segfaults

Post by 7im »

Hello jaymode, welcome to the forum.

The client should automatically move on to the next WU after 3 or more failures. If not, stop the client, and delete the Work folder, queue.dat file, and unitinfo.txt file. Then restart the client, you should get a new WU.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
jaymode
Posts: 4
Joined: Fri Apr 04, 2008 12:53 am

Re: 2619 segfaults

Post by jaymode »

That did not seem to work for me. I did that and it still came back with a 2619 WU.

Code: Select all

jmodi@JayDesktop:~/folding$ rm -r work
jmodi@JayDesktop:~/folding$ rm queue.dat
jmodi@JayDesktop:~/folding$ rm unitinfo.txt
jmodi@JayDesktop:~/folding$ ./fah6 -smp -forceasm

Note: Please read the license agreement (fah6 -license). Further
use of this software requires that you have read and accepted this agreement.

2 cores detected


--- Opening Log file [April 4 13:11:46]


# SMP Client ##################################################################
###############################################################################

                       Folding@Home Client Version 6.01beta2

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: /home/jmodi/folding
Executable: ./fah6
Arguments: -smp -forceasm

Warning:
 By using the -forceasm flag, you are overriding
 safeguards in the program. If you did not intend to
 do this, please restart the program without -forceasm.
 If work units are not completing fully (and particularly
 if your machine is overclocked), then please discontinue
 use of the flag.

[13:11:46] - Ask before connecting: No
[13:11:46] - User name: jaymode (Team 49191)
[13:11:46] - User ID: 16491510550E95BB
[13:11:46] - Machine ID: 1
[13:11:46]
[13:11:46] Work directory not found. Creating...
[13:11:46] Could not open work queue, generating new queue...
[13:11:46] - Preparing to get new work unit...
[13:11:46] + Attempting to get work packet
[13:11:46] - Connecting to assignment server
[13:11:47] - Successful: assigned to (171.64.65.56).
[13:11:47] + News From Folding@Home: Welcome to Folding@Home
[13:11:47] Loaded queue successfully.
[13:12:10] + Closed connections
[13:12:10]
[13:12:10] + Processing work unit
[13:12:10] Core required: FahCore_a2.exe
[13:12:10] Core found.
[13:12:10] Working on Unit 01 [April 4 13:12:10]
[13:12:10] + Working ...
[13:12:10]
[13:12:10] *------------------------------*
[13:12:10] Folding@Home Gromacs SMP Core
[13:12:10] Version 1.91 (2007)
[13:12:10]
[13:12:10] Preparing to commence simulation
[13:12:10] - Ensuring status. Please wait.
[13:12:12]
[13:12:12] Project: 2619 (Run 2, Clone 795, Gen 0)
[13:12:12]
[13:12:12] Assembly optimizations on if available.
[13:12:12] Entering M.D.
[13:12:32]  on if available.
[13:12:32] Entering M.D.
NNODES=4, MYRANK=1, HOSTNAME=JayDesktop
NNODES=4, MYRANK=3, HOSTNAME=JayDesktop
NNODES=4, MYRANK=0, HOSTNAME=JayDesktop
NNODES=4, MYRANK=2, HOSTNAME=JayDesktop
NODEID=3 argc=14
NODEID=1 argc=14
NODEID=2 argc=14
NODEID=0 argc=14
                         :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                 :-)  VERSION 3.3.99_development_20070720  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2006, The GROMACS development team,
            check out http://www.gromacs.org for more information.

         This program is free software; you can redistribute it and/or
          modify it under the terms of the GNU General Public License
         as published by the Free Software Foundation; either version 2
             of the License, or (at your option) any later version.

                                :-)  mdrun  (-:

Reading file work/wudata_01.tpr, VERSION 3.3 (single precision)
Note: tpx file_version 40, software version 48
starting mdrun 'TETHERED VESICLES'
125000 steps,   5000.0 ps.
[13:12:58] Completed 10 out of 125000 steps  (0%)
[cli_1]: aborting job:
Fatal error in MPI_Wait: Error message texts are not available
[0]0:Return code = 0, signaled with Aborted
[0]1:Return code = 1
[0]2:Return code = 0, signaled with Quit
[0]3:Return code = 0, signaled with Quit
[13:13:17] CoreStatus = 1 (1)
[13:13:17] Client-core communications error: ERROR 0x1
[13:13:17] Deleting current work unit & continuing...
uncle_fungus
Site Admin
Posts: 1288
Joined: Fri Nov 30, 2007 9:37 am
Location: Oxfordshire, UK

Re: 2619 segfaults

Post by uncle_fungus »

If you're running the client with "Advanced scientific methods" turned on, try turning it off in the configuration.
Post Reply