RAM usage Windows and Linux

A forum for discussing FAH-related hardware choices and info on actual products (not speculation).

Moderator: Site Moderators

Forum rules
Please read the forum rules before posting.

RAM usage Windows and Linux

Postby foldy » Sun May 03, 2020 9:10 pm

With Corona-Virus GPU work units I see RAM usage upto 1.5GB on Windows (not video ram). Does Linux have the same RAM usage or less/more? Can the FahCore RAM partially be swapped to disk or would that slow down the work unit processing? I ask because I want to calculate the minimum RAM needed for an 8 GPUs rig.
foldy
 
Posts: 1938
Joined: Sat Dec 01, 2012 4:43 pm

Re: RAM usage Windows and Linux

Postby JohnChodera » Sun May 03, 2020 9:14 pm

@foldy: Can you give some RAM usage estimates for specific projects?

We're not entirely sure whether this RAM usage is due to system size or written frame size. We're trying a few things that will reduce written frame sizes, so it will be helpful to see if that impacts the RAM usage, or if we have to do more restructuring of core22 code to minimize RAM usage for large projects.

~ John Chodera // MSKCC
User avatar
JohnChodera
Pande Group Member
 
Posts: 311
Joined: Fri Feb 22, 2013 10:59 pm

Re: RAM usage Windows and Linux

Postby foldy » Sun May 03, 2020 9:34 pm

Project 14432 (0, 545, 37) for GPU on Windows I see 475 MB private RAM usage.

I will report for other projects on Windows.

Is there a limit per GPU work units which FAH wants to hold? Like max 1 GB RAM per GPU work unit?

What are the numbers for CPU work units? Does RAM usage depend on used CPU core count?
foldy
 
Posts: 1938
Joined: Sat Dec 01, 2012 4:43 pm

Re: RAM usage Windows and Linux

Postby JohnChodera » Sun May 03, 2020 10:07 pm

The CPU cores are based on gromacs, so behavior may be totally different. We'd need to get the CPU core devs involved in order to sort that out.

For GPU cores, there's no limit on the WU size other than what can fit on a typical GPU, which can be pretty big. But we'd obviously love to minimize system memory footprint if possible to avoid causing problems with system operation.

~ John Chodera // MSKCC
User avatar
JohnChodera
Pande Group Member
 
Posts: 311
Joined: Fri Feb 22, 2013 10:59 pm

Re: RAM usage Windows and Linux

Postby PantherX » Mon May 04, 2020 5:51 am

I have noticed that Project 14561 FahCore_22 version 0.0.2 is using about 2 GBs of RAM (not VRAM) on my system. While I don't have an issue with it (I have 31.9 GBs RAM).
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time

Welcome To The F@H Support Forum Ӂ Troubleshooting Bad WUs Ӂ Troubleshooting Server Connectivity Issues
User avatar
PantherX
Site Moderator
 
Posts: 6343
Joined: Wed Dec 23, 2009 10:33 am
Location: Land Of The Long White Cloud

Re: RAM usage Windows and Linux

Postby foldy » Mon May 04, 2020 7:44 am

Project 14431 (0, 1109, 61) for GPU on Windows: 575 MB private RAM usage

I guess it is only an issue on PCs with many GPUs. If you build a sort of mining rig with 8 GPUs on x1 risers, 8 thread CPU and how much RAM? If work units stay below 1 GB RAM then total 8 GB RAM is enough. Or if swapping some RAM to disk is no issue. But if more than 1 GB RAM is needed per GPU work unit then system total RAM must be 16 GB.
foldy
 
Posts: 1938
Joined: Sat Dec 01, 2012 4:43 pm

Re: RAM usage Windows and Linux

Postby Joe_H » Mon May 04, 2020 8:10 am

What was the peak RAM usage at startup? There have been reports of some projects taking a large amount of VM / RAM during the startup phase as the WU data was prepared to be sent to the GPU.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Joe_H
Site Admin
 
Posts: 6450
Joined: Tue Apr 21, 2009 5:41 pm
Location: W. MA

Re: RAM usage Windows and Linux

Postby PantherX » Mon May 04, 2020 8:47 am

On my system, I didn't measure the RAM at start-up. I measured it ad-hoc as the WU was folding and it remained like that for a while... more than 5% IIRC.
User avatar
PantherX
Site Moderator
 
Posts: 6343
Joined: Wed Dec 23, 2009 10:33 am
Location: Land Of The Long White Cloud

Re: RAM usage Windows and Linux

Postby foldy » Mon May 04, 2020 11:27 am

I guess startup peak RAM usage is not so important as it can be swapped temporarily.
foldy
 
Posts: 1938
Joined: Sat Dec 01, 2012 4:43 pm

Re: RAM usage Windows and Linux

Postby Joe_H » Mon May 04, 2020 5:31 pm

foldy wrote:I guess startup peak RAM usage is not so important as it can be swapped temporarily.


As long as you have enough swap space available. There have been reports of some of the larger WUs exceeding 5 GB of VM during startup, and not working because too little swap was available.
Joe_H
Site Admin
 
Posts: 6450
Joined: Tue Apr 21, 2009 5:41 pm
Location: W. MA

Re: RAM usage Windows and Linux

Postby Kebast » Mon May 04, 2020 8:17 pm

Windows 10 pro GPU x22 14253 at 2,012 MB. I'll try to check catch one at start up sometime.
Image
Ryzen 3800x 12T - 3xGTX970
Kebast
 
Posts: 359
Joined: Thu Aug 06, 2015 6:21 pm

Re: RAM usage Windows and Linux

Postby foldy » Tue May 05, 2020 7:19 am

14201 FahCore_22 Windows 1700 MB

14251 FahCore_22 v0.0.5 Windows 1350 MB. After pause and resume it has 1100 MB. After next checkpoint it is at 1350 MB again.

14253 FahCore_22 Windows 1700 MB.

16441 FahCore_22 v0.0.5 Windows 970 MB

16442 FahCore_22 v0.0.5 Windows 900 MB
Last edited by foldy on Wed May 20, 2020 8:28 pm, edited 6 times in total.
foldy
 
Posts: 1938
Joined: Sat Dec 01, 2012 4:43 pm

Re: RAM usage Windows and Linux

Postby MeeLee » Tue May 05, 2020 8:51 pm

Since he said 'ram usage', I can presume that Windows by itself (without running anything else) uses up the bulk of this.
Perhaps 500-1000MB of RAM can be used for WUs, when using 1 or 2 GPUs, and the remainder (500-1000MB) could possibly be just Windows...

I'm not sure about Linux VS Windows, but since WUs are sent to both Linux and Windows, we can quite safely say that they (the WUs) are going to use the same amount of RAM on either OS.
It's just system overhead is a lot more on Windows.
MeeLee
 
Posts: 930
Joined: Tue Feb 19, 2019 11:16 pm

Re: RAM usage Windows and Linux

Postby Endgame124 » Tue May 05, 2020 8:54 pm

Joe_H wrote:
foldy wrote:I guess startup peak RAM usage is not so important as it can be swapped temporarily.


As long as you have enough swap space available. There have been reports of some of the larger WUs exceeding 5 GB of VM during startup, and not working because too little swap was available.

I just caught one WU a few days ago that went up to 4.8GB which put me pretty far into swap before dropping down to 2.2GB. I setup a 8GB zram swap partition to give me some extra space and so I wouldn’t put wear on my SSD for most swapping that occurs. So far, so good.
Endgame124
 
Posts: 54
Joined: Sat Mar 28, 2020 2:22 am

Re: RAM usage Windows and Linux

Postby holstien » Tue May 05, 2020 9:38 pm

Yes! Memory usage on windows and linux is similar. On my linux box with 8 gigs of ram, folding on the CPU and 2 gpus has periodically paged the GUI out to swap. Takes a few minutes to log back in. Enabling zswap ended up being a free and effective work around.

It acts like the gpu workers need matched space in system ram.
Image
holstien
 
Posts: 8
Joined: Tue Apr 14, 2020 1:02 pm

Next

Return to FAH Hardware

Who is online

Users browsing this forum: No registered users and 2 guests

cron