Problem with 64 Core Machine

[Splynncryth]

Hi All,

Logs below...

Code: Select all

Note: Please read the license agreement (FAH6.34-win32-SMP.exe -license). Furthe
r
use of this software requires that you have read and accepted this agreement.

[b]32 cores detected
'mpiexec' is not recognized as an internal or external command,
operable program or batch file.[/b]


--- Opening Log file [September 25 18:12:44 UTC]


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.34

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Fah\Fah
Executable: C:\Fah\Fah\FAH6.34-win32-SMP.exe
Arguments: -smp 64 -bigbeta -bigadv -verbosity 9

[18:12:44] - Ask before connecting: No
[18:12:44] - Proxy: 10.1.2.119:8080
[18:12:44] - User name: Splynncryth (Team 2999)
[18:12:44] - User ID: 11B8E5145CEB4385
[18:12:44] - Machine ID: 1
[18:12:44]
[18:12:44] Loaded queue successfully.
[18:12:44]
[18:12:44] + Processing work unit
[18:12:44] Core required: FahCore_a5.exe
[18:12:44] - Autosending finished units... [September 25 18:12:44 UTC]
[18:12:44] Core found.
[18:12:44] Trying to send all finished work units
[18:12:44] + No unsent completed units remaining.
[18:12:44] - Autosend completed
[18:12:44] Working on queue slot 01 [September 25 18:12:44 UTC]
[18:12:44] + Working ...
[18:12:44] - Calling '.\FahCore_a5.exe -dir work/ -nice 19 -suffix 01 -np 64 -ch
eckpoint 15 -verbose -lifeline 4020 -version 634'

[18:12:44]
[18:12:44] *------------------------------*
[18:12:44] Folding@Home Gromacs SMP Core
[18:12:44] Version 2.27 (Mar 12, 2010)
[18:12:44]
[18:12:44] Preparing to commence simulation
[18:12:44] - Ensuring status. Please wait.
[18:12:54] - Looking at optimizations...
[18:12:54] - Working with standard loops on this execution.
[18:12:54] - Previous termination of core was improper.
[18:12:54] - Files status OK
[18:13:03] - Expanded 25469175 -> 31941441 (decompressed 125.4 percent)
[18:13:03] Called DecompressByteArray: compressed_data_size=25469175 data_size=3
1941441, decompressed_data_size=31941441 diff=0
[18:13:04] - Digital signature verified
[18:13:04]
[18:13:04] Project: 2686 (Run 6, Clone 9, Gen 159)
[18:13:04]
[18:13:04] Entering M.D.
[18:13:10] Mapping NT from 64 to 64
[18:13:16] Completed 0 out of 250000 steps  (0%)


The bold problem above...
32 cores detected
'mpiexec' is not recognized as an internal or external command,
operable program or batch file.

Any idea why?

Also, I have tried running 2x F@h with 32 core each and it performs much faster than the one with 64 core with one F@H running.

My specifications is:
Dell R910
4x Intel Xeon 7555
128GB Memory
16x HDD SAS 300GB Raid-5
Windows Server 2008 R2 (Using this for Database with SQL Server 2008 R2)

Thank you...
-Splynt

[Jesse_V]

I believe the most threads the SMP can do is 32. So, make another SMP and run that with -smp 32 and it should use all 64 cores.
Are you sure you want the bigbeta flag? I'm not positive what it does but it doesn't seem like a common flag.

EDIT: also, don't worry about the "mpiexec' is not recognized as an internal or external command, ..." message. It has appeared on my machine ever since I installed SMP and it doesn't seem to mean or affect much. Probably some legacy code thing or something. Same with the "working with standard loops in this execution".

[jimerickson]

if you are running bigbeta this needs to be posted in the beta team forum if you are a beta team member.

[orion]

Don't use the bigbeta flag along with the bigadv flag. All bigbeta's have been moved to bigadv anyway so only use the bigadv flag.

I'm not an Intel guy but could HT be turned off in the BIOS or by the OS?

[Jesse_V]

Don't use the bigbeta flag along with the bigadv flag, all bigbeta's have been moved to bigadv anyway so only use the bigadv flag.

Concur. Bigadv should be sufficient unless you specifically want to test beta WUs, in which case you should be a beta team tester and post problems in their forums.
Thanks. Awesome machine by the way! I'm sure your graphics card is pretty sweet too...

[Splynncryth]

Thanks! I guess 32 is the limitation... Will setup another folder for the other 32.

I've check several other forums and it seems that HT gives additional boost in folding. But I'm quite sure I've seen the options to disable it in BIOS (not OS).

Noup, I'm not a beta-tester, I guess it's habbit when moving from linux -bigbeta folding to windows, tend to have that typo.

Graphics not in yet, 3x GTX590 is on the way ... for multi monitor setup.

We're planning to up the memory to 1-2 TB by early next year to host In-memory processing for our database. It's a baby project to drive business Intelligence platform for the company. (This is a demo setup). If it's successful, we'll have more budget next year to throw in (~350k USD) and might get a stronger server than this one ...

[orion]

32 threads isn't the limit for bigadv.

I and others have had no problem running them on our 48 core systems.

mine, viewtopic.php?f=55&t=17730&start=105#p192918

and one similar yours viewtopic.php?f=55&t=17730&start=90#p185039

[7im]

Core detection code defaults to 32 as a max, even though you have more cores (it's a legacy code issue). However, that is not how many threads the client is using. See this line in the log...

[18:13:10] Mapping NT from 64 to 64

And you should be able to confirm FAH is using 64 threads easily in task mangler.

[Splynncryth]

Much testing shows that in Windows, running 32 core with 2 f@h is far more efficient than 1 with 64 core. However, it is different in Linux. Looks like it's far more optimized running MP with Linux than in Windows. I wonder if the new v7 core will have this fix on Win. Any idea anyone?

I've slipped through VirtualBox once to see how it performs running Ubunto 11.04 & 11.10 in VB 4.1.2. However with 32 Core processing, 6903 is doing around 105 minutes per frame. Looks a little ridiculous to me .. Any idea why it performs this bad?

[Jesse_V]

As the forums can get pretty serious a times, so let me just point out
that I think your best solution at this point
is to simply put your computer in Narnia.
xkcd comic:
Best of luck.

[orion]

Does task manager show all 64 threads as 7im suggested?

Does CPUZ show the CPU's at their rated speed while folding?

[Splynncryth]

Yup, all cores is processing and showing maxed out. It's working, but 2x 32 core folds with 20k-30k points higher than 1x 64 core.

CPUz shows the CPU working at the said speed... actually higher, almost 2.0Ghz compared to the standard 1.86Ghz.

I'm trying tonight VirtualBox 4.1 with Ubuntu 10.10. Everyone on this setup seems to get optimal results. I've tried 4.1.2 with Ubuntu 11.10 & 11.04, but it didn't work. It tends to folds slower than normal. Over 100mins per frame.

Will update my findings tonight on the test.

[Splynncryth]

No go.... VB 4.1 showing funny error and it wont even load. Heh...

Doing more research on other options..

[7im]

...
It's working, but 2x 32 core folds with 20k-30k points higher than 1x 64 core.

Please be more specific how you arrived at these PPD numbers, because 1x64 does typically get better PPD than 2x32.

Or please post the Project # and frame times when running 1x64, and both when running 2x32. Thanks.

[MOBAJOBG]

Yup, all cores is processing and showing maxed out. It's working, but 2x 32 core folds with 20k-30k points higher than 1x 64 core.

CPUz shows the CPU working at the said speed... actually higher, almost 2.0Ghz compared to the standard 1.86Ghz.

I'm trying tonight VirtualBox 4.1 with Ubuntu 10.10. Everyone on this setup seems to get optimal results. I've tried 4.1.2 with Ubuntu 11.10 & 11.04, but it didn't work. It tends to folds slower than normal. Over 100mins per frame.

Will update my findings tonight on the test.

Hi!, Splynncryth. If you're using VirtualBox 4.1.2, under system settings "Processor" tab ...you can select up to a max of 32 processors per VM session only. Did you manage to configure it successfully? If everything goes smoothly, in Windows Task Manager, you should see an almost 100% activity while in Ubuntu 11.04 System Monitor, it should be almost 3200.

Since we're folding for the same team, just contact me in our home-based site if you wish for me to assist you in the setup which consist of VirtualBox 4.1.2 and Ubuntu 11.04 with ext4 filesystem. I've checked that you should be able to expect ~20min 30sec tpf for P6903 in a 32core VM session which you can create two of such.