Page 1 of 2

My Xeon just launched! :D

Posted: Tue Dec 25, 2018 9:08 pm
by ProDigit
My server just got online,
The Xeon E5 2690 v2 CPU, 10 cores 20 threads; works best with 18 threads configured on the desktop client.
All 10 CPU cores run at 100% load; and produce roughly 33-35k PPD in Windows 10 (Home 64-bit).

I only have 1 graphics card connected to it, which is the GTX 1050, on a PCIE 16x slot, which does about 133k PPD
Overall I'm hovering around 170k PPD.

I dropped the client form 19 CPU threads, to 18, because the CPU load was still 100%; but at 17 threads, few CPU cores hover around 75%.
I heard a graphics card takes up 1 core, but could it be that my graphics card fully uses 2 CPU threads?

The whole setup was purchased because I was in need for some more CPU power and RAM than my small desktop.
But since I won't use the server over 80% of the time, in the mean time, I figured it could do some FAH crunching!

I'm surprised to see the whole server only consumes just over 200 Watts on the wall, which is about 150 Watts below what I expected!
I do have a 500W PSU (80% Bronze), so I still have a good 200 Watts I could add, without taxing it.

Initially I was thinking of adding a lower end graphics card, to boost up the numbers; but I'm reevaluating that thought, due to much lower power consumption than anticipated.
I would want to stick with NVidia, because I already have an Nvidia card, but the 1060 just isn't worth it's price; it's almost 20% more expensive than a Radeon RX 570, and twice as expensive as a 1050.
For now, I'll be running it with only 1 card, until prices come down, and I'm ready to buy a second, more powerful one, in the 125-180W range.

Re: My Xeon just launched! :D

Posted: Tue Dec 25, 2018 9:50 pm
by toTOW
What project is running on the CPU ? A4 or A7 core ?

Your 1050 is doing fine :)

Re: My Xeon just launched! :D

Posted: Tue Dec 25, 2018 10:05 pm
by ProDigit
I think it was the A7, I'm not in the location right now; but I'm fairly sure it was A7 (and perhaps A5 for graphics?)

Re: My Xeon just launched! :D

Posted: Tue Dec 25, 2018 10:19 pm
by foldy
I heard a graphics card takes up 1 core, but could it be that my graphics card fully uses 2 CPU threads?
GPU only uses 1 CPU thread but you want to also keep 1 CPU thread for operating system - so 18 threads from 20 threads looks good.

Re: My Xeon just launched! :D

Posted: Tue Dec 25, 2018 10:37 pm
by ProDigit
On my Xeon system, it seems that the workload is shared across all cores.

It's not like I have 1 or 2 cores idling, or running low %.
They're all at 100% with 18 threads (on a 10 core, 20 thread CPU).

Come to think of it, there might have been some background activity, like installing updates, going on.
It'll take a while before I'll find the fastest way to fold on this machine.
Probably Linux, if possible Clear Linux, tiny core, or another lightweight distro.
I love Mint, but like Ubuntu, it's super popular, and has many unnecessary background activity tasks and programs, that all eat resources.

I think, as long as FAH works fine, I'd want to get an as barebone Linux as possible.
If it will increase score by as much as 5%, it'll be worth it!

Re: My Xeon just launched! :D

Posted: Sat Dec 29, 2018 7:28 am
by ProDigit
The CPU cores on my Xeon are running at 1,89Ghz under full load (2,49Ghz under low load).
I guess the graphics card needs more than just 1 CPU thread;
It appears to me it's using 1 thread, and 50-75% of the second hyperthreading core.

The 1050 is running between 110k PPD and 133k PPD; depending on which project.
In Taskmanager, I can see that Compute_0 is running at 95% load constantly.
There is very little 3D load on the card from the desktop.

While FAH runs fine under Windows, I'm currently testing out Puppy Linux, a smaller version of Ubuntu.
I still need a lot of time to make everything work, but Folding is working fine under Ubuntu.

Re: My Xeon just launched! :D

Posted: Sun Dec 30, 2018 12:35 am
by ProDigit
Just added my second card,

GTX 1050 Ti gets 83k PPD on the PCIE 16x slot, using 400MB of VRAM
This is significantly lower than the 133k PPD I regularly got with just the 1050 in place without any other card.

GT 1030 passive, gets only 38k PPD on a PCIE 1x slot with Riser, with only 100MB of VRAM usage.

Both graphics cards hover around 66C temps.

18 out of 20 threads on the Xeon CPU get 22k PPD.
Total memory usage 2,8GB of RAM


Does those numbers sound about right for the hardware?
I get a feeling my overall PPDs dropped by adding a second graphics card, not rose? :/

From 177k PPD to 166k PPD???
Makes no sense...

Edit: Oh, yes, forgot this is the first run with the second GPU.
I'm just hoping I configured the client correctly (setting CPU to 2 threads from max, and adding another entry for a GPU set to -1).

Re: My Xeon just launched! :D

Posted: Sun Dec 30, 2018 1:11 am
by bruce
There will be some variation between projects. Your report would be more meaningful if you indicated what project was running == and for that matter, if you took a averate across a number of WUs on each GPU.

Depending on you MB, your 16x slot might have reconfigured itself into two 8x slots when you added the second GPU (or something else) even though your only using 1x. I'm strictly guessing that your 1050 might be running slower at 8x than at 16x. In fact, that might be an insignificant change == or maybe you are comparing a different WU with entirely different characteristics or maybe you didn't let run long enough to stabilize. (The PPD of a newly downloaded WU needs to complete several checkpoints to establish a meaningful average.)

Re: My Xeon just launched! :D

Posted: Sun Dec 30, 2018 1:21 am
by ProDigit
Yes, it just needed time to level off.
the 1050 is now at 141k PPD (probably will drop a few PPDs later)
The 1030 is still at 38k (probably will still rise some)
And the CPU is at 23k PPD (I've always seen around this number).

Overall PPD rose to a more acceptable 202k PPD, which is in line with my prior points of 177k PPD + the new ~23k PPD 1030.

I'm slightly disappointed in the 1030, despite it having the same GPU and RAM speed as the 1050; and should have gone with a much higher card.
It'll take time before I can order another card.

Re: My Xeon just launched! :D

Posted: Sun Dec 30, 2018 4:33 am
by Joe_H
The 1030 does not use the same GPU chip as a 1050 Ti, it has half the shaders and runs at a slower clock speed. Essentially it has less than half the Single precision processing power of the 1050 Ti. But the 1030 does that using 30 W of power in comparison to 75 W used by the 1050 Ti.

Re: My Xeon just launched! :D

Posted: Sun Dec 30, 2018 10:24 am
by ProDigit
I've been able to snatch a 1060 card off of eBay, second hand, for about the same price as a new 1050, and will be installing it soon, when it arrives.

I've read this great article on WordPress about PPDs and power consumption:
https://greenfoldingathome.com/wp-comments-post.php

And it made me think,

System on 24/7, without folding =~25-30W =0PPD, $3 on electricity/ month
CPU folding =~100W = ~23k PPD, $10/ mo
CPU + 1050 = ~170W =~177k PPD, $15/mo
CPU + 1050 + 1030 =~200W =~200k PPD, $20/mo
CPU + 1050 + 1030 + 1060 = 320W. =~425k PPD, $35/mo

That's the added electricity cost, of the system folding alone, not including the cost of AC to cool down the place. We live in a fairly hot climate here in FL.

I've been thinking of disabling CPU folding, limiting myself to only GPU folding, to lower the electricity bill.
It should drop about 20-25% of the cost, and drop PPD by only 6-9%, plus, gets rid of the annoying constant CPU fan noise my wife is complaining about.
It at least makes her believe that the server isn't a vacuum-like energy hog, that it sounds like.

Doing this, I would be saving roughly $50 a year, at the cost of 23k PPD, taking in consideration running it 50% of the year as a space heater, or, running it an entire year, the AC costs would just eat through the profits, unless I find a way to leave the PC running on the balcony, outside in summer, which would be a workable solution! :o

I'd probably be hovering around an acceptable 250Watts total consumption, those 3 cards, no CPU; which is rather low for a space heater for winter.
But I hope to at some point, swap out the 1030 with a higher end model, before winter reallystarts here.
Gives me one more month or so, to determine what sub 250W card I can upgrade the 1030 with, and save up for it.

And to determine if in my situation, the rtx2070 is worth it over a GTX1080.
More than likely, initial purchase cost, and effectiveness of the risers and GPUs will be the main factor.
Also, if I am finding a workable solution, to split up the PCIE 16 port to 4 pcie 16 slots at 4x, without taking a performance hit, I could mount 3x extra GPUs outside of the PC casing.

Re: My Xeon just launched! :D

Posted: Sun Dec 30, 2018 9:49 pm
by bruce
Right now the weather here is rather cool. If you use electricity to heat in the Winter and to cool in the Summer, turn on CPU folding on in the winter and definitely turn it off in the summer. Using an electric heater that earns 0 PPD is not a good use of electricity.

Re: My Xeon just launched! :D

Posted: Mon Dec 31, 2018 3:56 am
by ProDigit
bruce wrote: Depending on you MB, your 16x slot might have reconfigured itself into two 8x slots when you added the second GPU (or something else) even though your only using 1x.
Turns out my Motherboard runs one slot at the full PCIE 16x 3, and the other 2 pcie 1x slots at 4x 2.0

When I look at bandwidth consumption on the PCIE 1x slot (using risers), the GT 1030 only uses 29%; which leads me to believe that the GTX 1050, being 3x the speed, could potentially be the fastest card that belongs in a PCIE 1x 4x slot.
The other slot is still working at the full 16x, and PCIE bandwidth speeds of the 1050 are recorded to be below measurable, at 1%.
Originally the Xeon supports a max of 40 PCI express lanes; and I presume that my motherboard caps it off at 16 + 4 + 4 = 24x total.

I'm currently running thermal and power consumption tests on Windows 10, on the desktop client.
I'll post the results later, but I did notice a few 'anomalies' possibly specific to my setup,
It appears that folding only with my 2 graphics cards, allows my CPU to run in turbo boost mode, and at a very low TDP of 161W.
Adding 1 CPU thread won't let the client do a WU on CPU, but adding 2 CPU threads, immediately adds 11% of CPU load (from 13% to 24%), and 7.5% of power at the wall to 173W; measured by a killawatt meter; but only nets me 4% on PPDs (from 176k PPD on 2 graphics cards, to 186k PPD on graphics cards + CPU); with the CPU throttling back from 1,93Ghz to it's base speed of 1.89Ghz.

So far adding threads from 2 to 8 cores in the desktop client, doesn't do anything to power consumption, or PPD points.
It might have to do with the desktop client needing to finish the WU first, before giving accurate results.
I'm still in the process of testing cores, and will update more later.

The aim is to reduce power consumption per work load, as well as reduce fan speed (tune the fan speed to only throttle up at 55-60C, and try keeping the CPU below this threshold); and taking fan speed, CPU load, and CPU and GPU temperature into consideration.
It definitely is an interesting study that's going to take a few days of diagnosis.

This paired with some cheap packaging tape modifications to the case, to reduce noise levels, and improve airflow and system cooling; been a busy and interesting few days!



Once this study is over, I'll probably only fold with GPUs, as I'm expecting a GTX1060 to be added soon, however in summer, I will put the server outside in the 90F balcony (thank God we have no sun there), to save on cooling cost.

Re: My Xeon just launched! :D

Posted: Mon Dec 31, 2018 11:29 pm
by bruce
ProDigit wrote:
bruce wrote:Turns out my Motherboard runs one slot .... and the other 2 pcie 1x slots at 4x 2.0
That's not possible. When the connection width is 1x and the signalling speed is 2.0, by not using all of the connections it can't possibly run faster than 1x 2.0
ProDigit wrote:I dropped the client form 19 CPU threads, to 18, because ...
That's really not necessary except to provide a more esthetic report from the program you're using to get activity reports. The FAHCore will do it for you. Look at the log from when you had 19 and you'll find it looks something like this: WARNING: AS lowered CPUs from 19 to 18. You may find other messages that further reduce it if, for example, there are no projects available that can be run with 18 threads.
Nevertheless, there's noting wrong with doing it yourself.

Re: My Xeon just launched! :D

Posted: Tue Jan 01, 2019 12:53 am
by ProDigit
GPUZ reports 4x speed on those slots.

I reduced it to 18 cores, because at 18 cores the CPU still was running at 100% load.
At 17 cores, one thread was around 75% active, the rest 100%; so I figured 19 cores plus overhead, plus the 1 core to feed the GPU might be overload.