Server advise!

A forum for discussing FAH-related hardware choices and info on actual products (not speculation).

Moderator: Site Moderators

Forum rules
Please read the forum rules before posting.
Post Reply
ProDigit
Posts: 242
Joined: Sun Dec 09, 2018 10:23 pm

Server advise!

Post by ProDigit »

Mod note: moved to appropriate forum - j

Hi guys,
I'm in the market for buying a portable, small, low power, cheap server.
Thinking something like an ARM, ~10 core @ ~2.0Ghz, ~150W server with 8-16GB of non-EEC RAM, preferably for under $1k.
I have it running off of Wifi, since I buy the server, mainly to replace my old dual core/4 threads 2,4Ghz Corei7 unit, and to save time compressing files locally, not over a network.
I don't think I'll need much more.

I have certain workloads that need to be done on my PC, and which I will do about 1 hour in a day (currently takes nearly an entire day).
On my idle time, I'd be running FAH program.

More than likely, I'll be running Linux.

I know most people use Graphics cards, but I'm looking at some sort of PC running either Linux or Windows, with a CPU with lots of cores.

So my questions are,
  1. Are CPU's recommended for FAH, or should I really better invest in GPUs?
  2. Is there anyone that can recommend some cheap servers (can be second hand, but preferably cheap new ones)?
  3. What kind of Linux is compatible with FAH?
Thanks!
JimboPalmer
Posts: 2573
Joined: Mon Feb 16, 2009 4:12 am
Location: Greenwood MS USA

Re: Server advise!

Post by JimboPalmer »

F@H Runs on x86-64 CPUs on Windows, MacOS and Linux.
F@H runs inside the Chrome Browser, but I think that also requires an x86-64 CPU.
F@H runs in some Android phones on ARM CPUs, basically if they resemble a Sony.
F@H can run on recent AMD and Nvidia graphics card in x86-64 Windows and Linux.

I would focus on GPUs if you want points. Spare CPUs might as well fold too.
Tsar of all the Rushers
I tried to remain childlike, all I achieved was childish.
A friend to those who want no friends
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Server advise!

Post by foldy »

I buy the server, mainly to replace my old dual core/4 threads 2,4Ghz Corei7 unit
You could use your old unit and add a fast GPU and run FAH on GPU 24/7. That would be the best bang for the buck.
Nvidia RTX 2070 for $500 with whole system using 250 watts max. But also a cheaper used GPU like GTX 1070 would fit.
ProDigit
Posts: 242
Joined: Sun Dec 09, 2018 10:23 pm

Re: Server advise!

Post by ProDigit »

That's a pity.

Been doing some research for servers,
They can be gotten quite cheap from the second hand market; businesses upgrading .

I was specifically looking for servers with ARM processors, due to power efficiency; and my only hints so far, were to buy AMD Opteron 6366 (16 cores), with a motherboard on the second hand market.
These cores go for about $60 each, great cores, but already 6 years old.
Their newer X1150, A1120/A1150/A1170 system on a chip cores have less CPUs though are more efficient, but are nowhere near the second hand market yet.

I also was looking into the Qualcomm Centriq 2400 series CPU, with 48 cores, which seemed very promising, however, the price for the CPU is well over my budget, and there are no 'full systems' out yet.

Aside from that, if I'd be limited to x86/x64, it would be Intel's Xeon processors. 6 cores, 12 threads are about the same performance and wattage as the AMD Opteron 6366 with 16 cores, but at double the price.

6 cores 12 threads is the max my budget will get for Xeon processors, plus any more than 8 cores or 6cores with 12 threads surpasses the desired thermal envelope of <100W; which would cost me $150 per year on electricity ($30-50 more than ARM servers of 85W).

A GPU is out of the question, since I need CPU processing power for my needs.
ProDigit
Posts: 242
Joined: Sun Dec 09, 2018 10:23 pm

Re: Server advise!

Post by ProDigit »

So I finally did the upgrade!
It's a great time to be buying server hardware!
$500 bought me a 12 core, 24 thread Intel Xeon processor 75W, 32GB of ECC DDR3 RAM at 1333Mhz, a 500W bronze PSU (80+%), a 256GB SATA6 SSD, MICROATX motherboard and case, windows operating system (still deciding which one, or I can save $100 on an OS and go Linux), and an NVidia GTX 1050.

Most powerful system I've ever owned, for under$500!
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Server advise!

Post by bruce »

How many PCIe slots are on your MICROATX?
ProDigit
Posts: 242
Joined: Sun Dec 09, 2018 10:23 pm

Re: Server advise!

Post by ProDigit »

I have 1 PCIE 16x and 2 x PCIE1 slots.
I bought the server to be small, like a half sized desktop, for the CPU, as I need CPU power for my own projects.
Adding a GPU was simply the right thing to do for FAH, but I don't want to raise our electric bill by too much.
I'm currently running at ~350Watts, which costs me at full load, for 24/7 folding, $350 extra on electricity per year.
I had the option to buy a motherboard for only $50 more, which could host 2x Xeon chips and had several PCIE slots.
But it would cost me too much on electricity per year.


So the system ended up costing me $530, as I still needed a CPU cooler.

Also, the GTX 960 is for sale between $50 and $75. The 1050 is for sale around $129. A similar card was sold nearly a year ago at $3k; this due to Bitcoin skyrocketing.
The market was basically raided for graphics cards that weren't affected by the bitcoin price spike, until supply met demand again, and now there's an oversupply on graphics cards.
JimboPalmer
Posts: 2573
Joined: Mon Feb 16, 2009 4:12 am
Location: Greenwood MS USA

Re: Server advise!

Post by JimboPalmer »

ProDigit wrote:I have 1 PCIE 16x and 2 x PCIE1 slots
So that server will feed one Graphics car for Folding.
Besides not fitting in a 1x slot, a 1x slot does not have enough bandwidth to run the Folding at home client. The rumor I hear is that you need at least 8x to run F@H on a GPU.
Tsar of all the Rushers
I tried to remain childlike, all I achieved was childish.
A friend to those who want no friends
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Server advise!

Post by bruce »

Rumor has it that 4x is enough to get some peroformace from a GPU, but less than full utilization, especially for higher performance GPUs. It's a trade-off.

Personally, I've never been able to make it work on a 1x slot. I'm not sure if it's the bandwidth or if the 1x slots are electrically incapable of supporting a GPU.

Some of the cyber-coin miners have mentioned that they've split a full performance slot down to run a number of separate risers but I'm not familiar
with the details or what else might be different except that once you accept that your GPUs will not get full performance, I think that multiple GPUs do
produce more results -- like maybe you can get 3 times 60% performance which is greater than 100% of one GPU.
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Server advise!

Post by foldy »

like maybe you can get 3 times 60% performance which is greater than 100% of one GPU.
Exactly
The pcie speed < pcie 3.0 x4 is only a big bottleneck on Windows with fast GPUs like gtx 1080. So using Linux or having a slower GPU like the gtx 1050 then it is running nearly full speed. On Linux you can even run fast GPUs like gtx 1080 ti on x1 risers with only 20% performance loss.
ProDigit
Posts: 242
Joined: Sun Dec 09, 2018 10:23 pm

Re: Server advise!

Post by ProDigit »

The PCIE 1x slot, has 250Mbits both way data transfer. That's 31MB/s. How is that slow?
The download package from FAH is only a few MB large.
What data is transferred over the bus?
Isn't it just a few megs of folding material, and a few megs of results?
I just don't understand why PCIE 1x slot would be considered too slow.
It's like, yeah, you'd have to run the client for 10 hours, to get the same results as a GTX1080 running for only 1 hour.
But it's still something, no?

Unless the folding project is running out of tasks.
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Server advise!

Post by bruce »

Data transferred over the PCIe bus has nothing to do with the size of the download package from FAH.

The GPU is effectively a remote co-processor using Stream Computing. Data is loaded into main RAM and formed into blocks (compute kernels) that can be processed by the GPU. There is a constant stream of new data being transferred to the GPUs VRAM and results being returned to main RAM. If fresh kernels don't arrive at the GPU as fast as it can process them, the GPU waits for more work.

As was suggested above, with an 8x (or 4x) slot, most modern GPUs can process the data at a similar speed as new requests can be transferred so performance is only slightly slower than if it is provided at 16x. If it's transferred across a 1x slot, the GPU does a lot more waiting than processing. As I said above, it's a trade-off. Do you want your throughput to be throttled by the number and speed of the shaders or by the bandwidth of the PCIe transfers or by a mixture of the two?
rwh202
Posts: 425
Joined: Mon Nov 15, 2010 8:51 pm
Hardware configuration: 8x GTX 1080
3x GTX 1080 Ti
3x GTX 1060
Various other bits and pieces
Location: South Coast, UK

Re: Server advise!

Post by rwh202 »

Driver and / or core inefficiencies mean that more data than necessary is being constantly transferred over the PCI bus on windows.

I've got a 1060 on windows and that shows 25% of a x16 gen2 interface. 1080 Ti on linux shows 1%.

Also slowdowns become apparent well before 100% saturation - don't know if that's latency or whatever.

The distribution of data across GPU and system memory might be something to look at within the code. Previously GPU memory was probably considered more restricted than system memory. Nowadays they are probably equivalent and on most of my GPU systems its 8GB system (50% used) and 24GB on the GPUs (1% used)...
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Server advise!

Post by bruce »

If you are watching a streaming video, the important issue is whether your ISP can supply data faster than your TV can consume it. e.g.- watching images from a HD broadcast might break up sooner than watching a lower resolution video. If you are connected by satellite, heavy rain will cause more problems than clear skies or even light rain. Double-buffering can certainly help because it gives the downloaded data stream more time to recover from small pauses.

Also, consider the fact that your OS (and maybe other processes) are attempting to get data to your GPU to update your screen during the intervals when the PCIe bus is not busy writing FAH kernels to the same GPU VRAM. (You can search this forum for "screen lag")
Post Reply