project:5771 run:1 clone:152

[Napoleon]

I seem to have a knack of getting/finding these kind-of-errorish thingies. This time it was on my Atom330 setup. FWIW, my c:\fah\data\work directory is excluded in the Avast settings.

06:39:01:WU01:FS00:0x11:Completed 100%
06:39:01:WU01:FS00:0x11:Successful run
06:39:01:WU01:FS00:0x11:DynamicWrapper: Finished Work Unit: sleep=10000
06:39:11:WU01:FS00:0x11:Reserved 75808 bytes for xtc file; Cosm status=0
06:39:11:WU01:FS00:0x11:Allocated 75808 bytes for xtc file
06:39:11:WU01:FS00:0x11:- Reading up to 75808 from "01/wudata_01.xtc": Read 75808
06:39:11:WU01:FS00:0x11:Read 75808 bytes from xtc file; available packet space=786354656
06:39:11:WU01:FS00:0x11:xtc file hash check passed.
06:39:11:WU01:FS00:0x11:Reserved 15168 15168 786354656 bytes for arc file=<01/wudata_01.trr> Cosm status=0
06:39:11:WU01:FS00:0x11:Allocated 15168 bytes for arc file
06:39:11:WU01:FS00:0x11:- Reading up to 15168 from "01/wudata_01.trr": Read 15168
06:39:11:WU01:FS00:0x11:Read 15168 bytes from arc file; available packet space=786339488
06:39:11:WU01:FS00:0x11:trr file hash check passed.
06:39:11:WU01:FS00:0x11:Allocated 560 bytes for edr file
06:39:11:WU01:FS00:0x11:Read bedfile
06:39:11:WU01:FS00:0x11:edr file hash check passed.
06:39:11:WU01:FS00:0x11:Allocated 0 bytes for logfile
06:39:11:WU01:FS00:0x11:Could not open/read logfile=<01/wudata_01.log>; Cosm status=-1
06:39:11:WU01:FS00:0x11:GuardedRun: success in DynamicWrapper
06:39:11:WU01:FS00:0x11:GuardedRun: done
06:39:11:WU01:FS00:0x11:Run: GuardedRun completed.
06:39:15:WU01:FS00:0x11:+ Opened results file
06:39:15:WU01:FS00:0x11:- Writing 92048 bytes of core data to disk...
06:39:15:WU01:FS00:0x11:Done: 91536 -> 90321 (compressed to 98.6 percent)
06:39:15:WU01:FS00:0x11: ... Done.
06:39:15:WU01:FS00:0x11:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
06:39:15:WU01:FS00:0x11:Shutting down core
06:39:15:WU01:FS00:0x11:
06:39:15:WU01:FS00:0x11:Folding@home Core Shutdown: FINISHED_UNIT
06:39:16:WU01:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
06:39:16:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:5771 run:1 clone:152 gen:2217 core:0x11 unit:0x5d10947650df62ed08a900980001168b
06:39:16:WU01:FS00:Uploading 88.70KiB to 171.67.108.11
06:39:16:WU01:FS00:Connecting to 171.67.108.11:8080
06:39:18:WU01:FS00:Upload complete
06:39:18:WU01:FS00:Server responded WORK_ACK (400)
06:39:18:WU01:FS00:Cleaning up

Currently I'm monitoring all but FAH* processes on my Atom330 to see if I can catch any illegitimate process(es) tampering with the work directory. Got to drop unfiltered events from the log though, otherwise it would swamp my system pretty quickly:


The System process (PID 4, OS kernel) seems to do something extra during checkpointing, but I'm assuming this kind of stuff is legitimate, file timestamps being updated etc:


For the past couple of hours, no news if I temporily filter out System from the recorded event log:


Now, if I happen to catch any 3rd party processes (other than FAH*) having messed around in the work folder when the FAH log shows weird things, it's a clear-cut case. But if I see only System process, what then? I think it highly unlikely that my OS kernel has been corrupted by malware. As for legitimate OS kernel doing something it shouldn't be doing with FAH files - aww, come on...

Is the boldfaced thingy in the log excerpt just some cosmetic issue? How about the rest of you, are you seeing "ghosts" like this? Buy me a ticket out of Paranoia city, please.

[bollix47]

It appears to have credited properly:

Hi ION (team 191980),
Your WU (P5771 R1 C152 G2217) was added to the stats database on 2012-12-29 23:04:51 for 353 points of credit.

[Napoleon]

Yep, that's me. Once upon a time I set up a team (191980) just for myself in order to differentiate between various bits of my folding. Zotac430 is by far the biggest producer in the "team". You'll never guess where that name came from...

Anyway, the client uploaded the WU to server with NO_ERROR status so it makes sense that it got full credit. I don't think the log file contains any information relevant to the science, but anomalies like these get me worried about data corruption going unnoticed. Like I said, Paranoia city.

[Napoleon]

Now that I had a clue what to look for, my old log files actually have a bunch of identical "cosm status=-1" reports. All FahCore_11.exe and about wudata_01.log file. Doesn't always happen, but it isn't an isolated incident either.

[bruce]

I'm going to venture a guess:

What are the chances that you're running out of space on your folding drive and some file isn't getting written?

There's no universal list of files that will be created by an arbitrary core using whatever GROMACS settings were configured for that project, but I can't imagine that wudata_01.log wouldn't have been created and gradually extended as the WU progressed. Where did it go?

[Napoleon]

This is the system drive (C:) on the Atom330 setup, so it has sufficient space available. If it occasionally ran out of space just like that, I assume that the warnings & errors I'd see would be a great deal more obvious than a few strange lines in a FAH log every now and then.

I've recently started to run FahCore_11 & uniproc work on a 32bit WinXP Home setup as well (the 9400 GT graphics card is very similar to the integrated ION GPU). It's a fresh install and fully dedicated to folding, so it has no extra SW. I haven't seen "cosm status=-1" stuff there, at least not yet. I'll keep my eyes peeled in case this occurs on the XP rig as well.

[Napoleon]

Just a thought: could relatively frequent pause/resume operations have something to do with this? All this is coming from the ION GPU slot. Due to screen lag issues I need to pause it mid-WU every so often. I very rarely pause the other slots, and I don't see anything like this with them.

[Napoleon]

Never mind my ION1 anymore. Folding with it has been a bit problematic lately... for example, the briefly active FahCore_15.exe v2.25 for pre-Fermi cards failed immediately on ION1 (9400M G), even though it worked on a 9400GT graphics card on a different machine. Specification wise the two GPUs should be nearly identical, and GPU2 folding performance is nearly identical too. But since I need to pause the ION slot quite frequently - and already suspect that the frequent mid-WU pauses may be causing intermittent problems - I figured I'll just switch the ION to Einstein@Home crunching once the current WU is finished. CPU & dedicated GPU3 folding on my Atom330 goes on, of course.