Project: 10125 (Run 53, Clone 0, Gen 120)

Moderators: Site Moderators, FAHC Science Team

Post Reply
cduchat
Posts: 12
Joined: Wed Aug 29, 2012 1:58 am

Project: 10125 (Run 53, Clone 0, Gen 120)

Post by cduchat »

Hello,

I am folding the WU 10125 (53, 0, 120) almost continuously since yesterday and the PPD is ridiculously low as compared to what it is usually (about 220 PPD when I am using a Web browser, about 280 PPD when I am not using the computer, whereas the PPD is usually higher than 5000 for this slot).

The TPF is actually over 2 hours, the core has not crashed and the log file is not showing any evidence of error... the client is just stating that it will need more than one week to finish this WU.

The task manager is showing that the different process of F@H are all there and not duplicated and that the folding core is well the main resource consuming one.

I already had this kind of strange behavior in the past but it was not with the same core (if I remember well it was two different projects 760X on core a4, whereas the project 10125 is using the core a3) and have not been able to fix that.

The involved configuration is: Intel Core i7-2720QM (mobile CPU @ 2.2GHz), Windows 7 pro 64bit, F@H v7.1.52 (there is also a GPU slot but according to the task manager it is consuming almost no CPU).

If you have any advice, it will be highly appreciated.

Best,

cduchat
PinHead
Posts: 285
Joined: Tue Jan 24, 2012 3:43 am
Hardware configuration: Quad Q9550 2.83 contains the GPU 57xx - running SMP and GPU
Quad Q6700 2.66 running just SMP
2P 32core Interlagos SMP on linux

Re: WU 10125 (53, 0, 120)

Post by PinHead »

Is your GPU ATI/AMD ? Or is it on battery vs AC?
cduchat
Posts: 12
Joined: Wed Aug 29, 2012 1:58 am

Re: WU 10125 (53, 0, 120)

Post by cduchat »

The GPU is a NVidia Quadro 1000M.

The laptop is only on AC (I stop F@H when I have to use the battery, what happens very rarely).
PinHead
Posts: 285
Joined: Tue Jan 24, 2012 3:43 am
Hardware configuration: Quad Q9550 2.83 contains the GPU 57xx - running SMP and GPU
Quad Q6700 2.66 running just SMP
2P 32core Interlagos SMP on linux

Re: WU 10125 (53, 0, 120)

Post by PinHead »

If running on v7, could you post your system log? If running on v6.xx, could you post your command line or shortcut line?

Guessing that either something else is running ( task manager will tell as FAH should be top by cpu usage ) or your not running -smp.
cduchat
Posts: 12
Joined: Wed Aug 29, 2012 1:58 am

Re: WU 10125 (53, 0, 120)

Post by cduchat »

Here is the log (only the parts concerning the slot and WU, and after resuming from a Windows hibernation):

03:11:46:WU02:FS01:Starting
03:11:47:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Christophe/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Cor ... ore_a3.exe -dir 02 -suffix 01 -version 701 -lifeline 1268 -checkpoint 15 -np 8
03:11:47:WU02:FS01:Started FahCore on PID 175012
03:11:48:WU02:FS01:Core PID:177116
03:11:48:WU02:FS01:FahCore 0xa3 started
03:11:48:WU02:FS01:0xa3:
03:11:48:WU02:FS01:0xa3:*------------------------------*
03:11:48:WU02:FS01:0xa3:Folding@Home Gromacs SMP Core
03:11:48:WU02:FS01:0xa3:Version 2.27 (Dec. 15, 2010)
03:11:48:WU02:FS01:0xa3:
03:11:48:WU02:FS01:0xa3:Preparing to commence simulation
03:11:48:WU02:FS01:0xa3:- Looking at optimizations...
03:11:48:WU02:FS01:0xa3:- Files status OK
03:11:48:WU02:FS01:0xa3:- Expanded 787984 -> 1476132 (decompressed 187.3 percent)
03:11:48:WU02:FS01:0xa3:Called DecompressByteArray: compressed_data_size=787984 data_size=1476132, decompressed_data_size=1476132 diff=0
03:11:48:WU02:FS01:0xa3:- Digital signature verified
03:11:48:WU02:FS01:0xa3:
03:11:48:WU02:FS01:0xa3:Project: 10125 (Run 53, Clone 0, Gen 120)
03:11:48:WU02:FS01:0xa3:
03:11:48:WU02:FS01:0xa3:Assembly optimizations on if available.
03:11:48:WU02:FS01:0xa3:Entering M.D.
03:11:54:WU02:FS01:0xa3:Using Gromacs checkpoints
03:11:54:WU02:FS01:0xa3:Mapping NT from 8 to 8
03:11:56:WU02:FS01:0xa3:Resuming from checkpoint
03:11:56:WU02:FS01:0xa3:Verified 02/wudata_01.log
03:11:56:WU02:FS01:0xa3:Verified 02/wudata_01.trr
03:11:56:WU02:FS01:0xa3:Verified 02/wudata_01.xtc
03:11:56:WU02:FS01:0xa3:Verified 02/wudata_01.edr
03:11:57:WU02:FS01:0xa3:Completed 356812 out of 2000000 steps (17%)
03:28:03:WU02:FS01:0xa3:Completed 360000 out of 2000000 steps (18%)


In terms of CPU usage, FahCore_a3.exe is on top of the list, with 80 to 90% of the CPU load.
P5-133XL
Posts: 2948
Joined: Sun Dec 02, 2007 4:36 am
Hardware configuration: Machine #1:

Intel Q9450; 2x2GB=8GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460; Windows Server 2008 X64 (SP1).

Machine #2:

Intel Q6600; 2x2GB=4GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460 video card; Windows 7 X64.

Machine 3:

Dell Dimension 8400, 3.2GHz P4 4x512GB Ram, Video card GTX 460, Windows 7 X32

I am currently folding just on the 5x GTX 460's for aprox. 70K PPD
Location: Salem. OR USA

Re: WU 10125 (53, 0, 120)

Post by P5-133XL »

I'll start with that the SMP cores do not like outside usage of the CPU at all. Running at 80-90% is not good. It should be very close to 100%. SMP folding is designed not to interfere with your useage of the machine so it runs at a very low priority. When an outside application uses a core, then because of the Windows priority system, the folding thread using that core gets suspended so as to not interfere with your useage. Unfortunately with SMP folding all the threads are very closely synchronized so that when one folding thread gets suspended, the others will just sit in a loop waiting for the suspended thread to restart. The result can be a tremendous increase in frame times and a drastic drop in PPD with a relatively modest external CPU usage. The change in PPD and frame times is not even close to linear to the external CPU usage.

Your CPU is a quad core and if a single core is being used externally then it will show as 75% folding and 25% other. You are showing 80-90% folding so on one of your cores folding is only running at 5%-15% and that will produce very large-scale folding performance drop. To regain folding performance, you need to deal with whatever that process is that is using your CPU whether it is intentional like a screen saver or unintentional like malware.
Image
cduchat
Posts: 12
Joined: Wed Aug 29, 2012 1:58 am

Re: WU 10125 (53, 0, 120)

Post by cduchat »

Thanks, but I don't think that can be the source of my problem: I am sometimes running a program doing very intensive usage of only one core (I am not using it actually) and even when this program is running, the PPD is not going that low. The only software running actually are F@H, Thunderbird and Firefox (these two are stopped at night, inducing a very limited change in PPD).

As said in my first post, it is the third or fourth time I have a WU behaving like that... but I am folding on this computer, with exactly the same usage of other software (mainly Firefox, Thunderbird, Word and Excel, and very rarely more CPU-expensive programs) since december and I really had time to figure out what is the impact of my usage of other sofware on the PPD.

The screen saver is usually not interfering with F@H and haven't modify it's parameters recently, so I don't see why it would be interfering specifically with this WU, and the computer is "clean" (maintained up-to-date, well protected, scanned on a weekly basis).
P5-133XL
Posts: 2948
Joined: Sun Dec 02, 2007 4:36 am
Hardware configuration: Machine #1:

Intel Q9450; 2x2GB=8GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460; Windows Server 2008 X64 (SP1).

Machine #2:

Intel Q6600; 2x2GB=4GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460 video card; Windows 7 X64.

Machine 3:

Dell Dimension 8400, 3.2GHz P4 4x512GB Ram, Video card GTX 460, Windows 7 X32

I am currently folding just on the 5x GTX 460's for aprox. 70K PPD
Location: Salem. OR USA

Re: WU 10125 (53, 0, 120)

Post by P5-133XL »

I was using the screen saver as an example, it can be anything. Matter of fact that particular example would be unlikely in you were looking at the task manager and a screen saver won't be running when you are doing something. As you using CPU intensive tasks, it doesn't take CPU intensive tasks to produce the folding performance drop. Most people wouldn't think a 10-20% usage to be CPU intensive and a single core being used is as bad as lots of cores because folding is synchronized so that the other folding threads are just waiting for that one thread to come back. Those numbers are plenty of outside usage to see frame times drop drastically. I'm just using your numbers ...

If you are looking at other things that could cause folding performance drops. Look to power settings and make sure that your notebook is not going into a low-power using mode when you are not actively using it. If the notebook drops the CPU's clock frequency to a very low number, to save power, then that could affect folding performance even without external CPU usage.
Image
cduchat
Posts: 12
Joined: Wed Aug 29, 2012 1:58 am

Re: WU 10125 (53, 0, 120)

Post by cduchat »

When I look at the task manager, there FahCore_a3.exe representing 80 to 90% of the load, the rest is a lot ot "small" processes but none of them is going higher than 2% (they are mostly so inexpensive that their load is marked as 0%). Sincel then, I think Windows may be configured to assing these "small" processes in a way that produces an even load on all the cores.

Concerning the power settings, I have configured a minimal power usage (via a low frequency) when on battery, but full power (and steady state at the highest frequency) when on AC power.

I have already tried to figure out by myself what could be the source of the problem, searching on the forum, checking all the settings and possible software conflicts, etc. before posting here. That's why I think the problem is really related to the WU and the way it is treated by FahCore, than to my computer settings.

I don't know if it is possible, but maybe instead of having 8 different threads (normal behavior for a SMP calculation on a Core i7), I might have 8 identical threads (as if each virtual core was running a single core client, all of them doing exactly the same calculations)?!?!?
P5-133XL
Posts: 2948
Joined: Sun Dec 02, 2007 4:36 am
Hardware configuration: Machine #1:

Intel Q9450; 2x2GB=8GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460; Windows Server 2008 X64 (SP1).

Machine #2:

Intel Q6600; 2x2GB=4GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460 video card; Windows 7 X64.

Machine 3:

Dell Dimension 8400, 3.2GHz P4 4x512GB Ram, Video card GTX 460, Windows 7 X32

I am currently folding just on the 5x GTX 460's for aprox. 70K PPD
Location: Salem. OR USA

Re: WU 10125 (53, 0, 120)

Post by P5-133XL »

0% processes don't matter they are not using the CPU. Try sorting the CPU usage column by clicking on the header. Something is using that 10-20%. Also make sure that the show processes from all users box is checkmarked.

If you can't get the CPU usage down, try folding using SMP:6 (Some WU's do not like SMP 7 and will fail because of it). That will allow some CPU usage to occur without slowing down folding too much. You will lose some folding performance but it tends to be minimal because the CPU cores that won't be used (by folding) tend to by the hyper-threaded cores rather than the real CPU cores.

You can access the SMP:6 setting in the advanced mode of FAHControl then go to configure and the slots tab. From there highlight the SMP slot and click edit. The SMP section will allow you to change the number of cores assigned to folding.
Image
cduchat
Posts: 12
Joined: Wed Aug 29, 2012 1:58 am

Re: WU 10125 (53, 0, 120)

Post by cduchat »

I just swiched to SMP:6 and I don't see any change after about half an hour.

The process using about 10-12% of the CPU is named "WmiPrvSE.exe", is described as "WMI Provider Host" and is used by the SYSTEM.

As I swiched to SMP:6, there is also a virtual process "System Idle Process" representing the idle cycles of the CPU, it is at about 8% (it was not there or at 0% before swiching).
cduchat
Posts: 12
Joined: Wed Aug 29, 2012 1:58 am

Re: WU 10125 (53, 0, 120)

Post by cduchat »

Update: the PPD starts increasing, apparently this WU is just a lot more sensible than the others to the presence of other running processes.

Thanks for your time and pieces of advice.
P5-133XL
Posts: 2948
Joined: Sun Dec 02, 2007 4:36 am
Hardware configuration: Machine #1:

Intel Q9450; 2x2GB=8GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460; Windows Server 2008 X64 (SP1).

Machine #2:

Intel Q6600; 2x2GB=4GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460 video card; Windows 7 X64.

Machine 3:

Dell Dimension 8400, 3.2GHz P4 4x512GB Ram, Video card GTX 460, Windows 7 X32

I am currently folding just on the 5x GTX 460's for aprox. 70K PPD
Location: Salem. OR USA

Re: Project: 10125 (Run 53, Clone 0, Gen 120)

Post by P5-133XL »

To make smp:6 effective, you do have to restart folding.

Wmiprvse.exe

The system idle is normal now that there are two cores not being used by folding.
Image
cduchat
Posts: 12
Joined: Wed Aug 29, 2012 1:58 am

Re: Project: 10125 (Run 53, Clone 0, Gen 120)

Post by cduchat »

I have restarted Folding immediately after changing the parameter.

Thanks for the link.
Post Reply