Guidance for building a folding dedicated computer

A forum for discussing FAH-related hardware choices and info on actual products (not speculation).

Moderator: Site Moderators

Forum rules
Please read the forum rules before posting.

Re: Please advise, hoping to build a dedicated folding machi

Postby ComputerGenie » Sun Jul 30, 2017 7:56 am

boristsybin wrote:...
4 to 7 1080Ti
...
watercooling

Now there's a good bit of wasted $. :e?:
User avatar
ComputerGenie
 
Posts: 242
Joined: Mon Dec 12, 2016 4:06 am

Re: Guidance for building a folding dedicated computer

Postby ollie1983 » Sun Jul 30, 2017 9:52 am

Ive got a water cooling loop in my existing PC. It is good because heat from the CPU is dumped straight out of the case and not recycled etc.

I will definitely look at the X79 Asus/SW platform. Thanks.
ollie1983
 
Posts: 7
Joined: Sat Jul 29, 2017 8:12 pm

Re: Guidance for building a folding dedicated computer

Postby Nathan_P » Sun Jul 30, 2017 10:56 am

I would go either x99 or wait for Threadripper (TR) to launch. Not only will you get more PCIe lanes with TR, (important if you want to get the maximum performance from your cards), it should be cheaper than an intel rig. Despite what people say PCIe speed does affect F@H performance and although in some cases its not much, 2-3% longer on a GTX1070/1080 is 6k-8k PPD less. GPU's would depend on budget but I would look at 1080 as a minimum.

Personally I would look at 12C/24t TR, high end mobo, upto 3 GPU's and a 1000-1200w PSU in a large case with lots of fans along with a cheap HDD and Linux as the OS, buy decent branded parts - especially Ram, Mobo and PSU. If you want to fold on both CPU and GPU I would go for 16C/32T TR or dual processor intel xeon or AMD EYPC. I'm currently running a CPU rig on dual 12c/24t xeon's and I'm getting anything from 250-380k PPD.

Once you have al the parts the only thing that may need to be upgraded are the GPU's as newer ones are faster/more power efficient but the base components should last 5-6 years
Image
Nathan_P
 
Posts: 1400
Joined: Wed Apr 01, 2009 9:22 pm
Location: Jersey, Channel islands

Re: Please advise, hoping to build a dedicated folding machi

Postby boristsybin » Sun Jul 30, 2017 11:10 am

ComputerGenie wrote:
boristsybin wrote:...
4 to 7 1080Ti
...
watercooling

Now there's a good bit of wasted $. :e?:
there was request for
ollie1983 wrote:a fairly brutish PC to fold with
Image
boristsybin
 
Posts: 50
Joined: Mon Jan 16, 2017 11:40 am
Location: Russia, Moscow

Re: Please advise, hoping to build a dedicated folding machi

Postby Aurum » Sun Jul 30, 2017 1:17 pm

ollie1983 wrote:...looking to build or buy a fairly brutish PC to fold with. It might get some VR gaming...
Hi Ollie, I don't game but I think there's a dichotomy in approaches. I think gaming benefits from a fast CPU RAM combination making an i7 look great or a Ryzen (more bang for the buck) look great, but these only have 24 PCIe lanes. Folding benefits from more cores (Xeon E5 with 40 PCIe lanes) and especially more PCIe lanes, not so much on RAM, but is often PCIe throughput limited. The best info I can recall to get into the nitty gritty is the thread foldy posted a couple back on PCIe bandwidth.
AMD MBs run all PCIe lanes through their Northbridge switch and this slows it down. Most Intel MBs route at least 8 lanes thru Northbridge. The only MBs I've found that route all 40 lanes directly from the CPU to the graphics card slots are the Gigabyte GA-X99 boards. Take a look at the overview web page and especially in the manual and find the block diagram of the MB. Where this matters is when you use four 1080 Ti's you can get x8x8x16x8 or with 3 cards x16x8x16 or x16x16x8.
I'm watching to see how MB makers implement the AMD Thread Ripper with 64 3.0 PCIe lanes. Sadly they always design these MBs for gamers and not scientists so no great expectations. I'd like to have either x16x16x16x16 or x8x8x8x8x8 directly from CPU to graphics cards and fewer bells 'n whistles and blinky lights.
http://wccftech.com/amd-ryzen-threadripper-1950x-cpu-performance-benchmarks-leak/
Last edited by Aurum on Sun Jul 30, 2017 1:41 pm, edited 1 time in total.
The Folding Fleet Image is a member of the Image team.
User avatar
Aurum
 
Posts: 281
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: Guidance for building a folding dedicated computer

Postby Aurum » Sun Jul 30, 2017 1:28 pm

When considering 5 to 7 GPU cards keep in the mind the total at-the-wall power consumption of a rig. I'm in the US and most homes have 120 V 15 amp circuit breakers. I wired in additional 20-amp breakers to supply my folding rigs. I can put one 4-GPU rig on a 15-amp breaker or two 4-GPU rigs on a 20-amp breaker. You may have more head room in 240 Volt countries but do the math and be safe as this can be a serious fire hazard.
User avatar
Aurum
 
Posts: 281
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: Guidance for building a folding dedicated computer

Postby Aurum » Sun Jul 30, 2017 1:40 pm

Nathan_P wrote:...the base components should last 5-6 years.
I predict this is not true of the M.2 SSD cards that are buried under the second graphics cards on all MBs I've seen. Why on Earth any competent engineer would allow that design layout is beyond me. That's a blackbody cavity and means the M.2 SSD will operate at 66C to 80C. Heat exponentially accelerates electromigration and electronic failure. I'll be amazed if they make it to 2 years under these severe conditions. Also if you overload your PSU you'll shorten its life as well. I only mention PSU because I can't forget some of the cryptominers I've read that bragged about how they got 1,300 Watts at the wall on a 1,200 Watt PSU.
User avatar
Aurum
 
Posts: 281
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: Guidance for building a folding dedicated computer

Postby boristsybin » Sun Jul 30, 2017 4:31 pm

Aurum wrote:got 1,300 Watts at the wall on a 1,200 Watt PSU
That`s normal for 87+ gold certified PSU. 1200/0.87=1379 :)
boristsybin
 
Posts: 50
Joined: Mon Jan 16, 2017 11:40 am
Location: Russia, Moscow

Re: Guidance for building a folding dedicated computer

Postby Nathan_P » Sun Jul 30, 2017 8:09 pm

Aurum wrote:
Nathan_P wrote:...the base components should last 5-6 years.
I predict this is not true of the M.2 SSD cards that are buried under the second graphics cards on all MBs I've seen. Why on Earth any competent engineer would allow that design layout is beyond me. That's a blackbody cavity and means the M.2 SSD will operate at 66C to 80C. Heat exponentially accelerates electromigration and electronic failure. I'll be amazed if they make it to 2 years under these severe conditions. Also if you overload your PSU you'll shorten its life as well. I only mention PSU because I can't forget some of the cryptominers I've read that bragged about how they got 1,300 Watts at the wall on a 1,200 Watt PSU.


You do not need an M2 ssd card for an F@H rig, a normal HDD from WD or Seagate et al is perfectly adequate. M2 is a pointless expense. PSU wise 3x 1080Ti is 750w TDP, add 130w for your cpu & mobo and a 1000-1200w PSU is still more than enough.

A 1070 and 1080 + CPU, Mobo, HDD and fans only pulls 450w from the wall.

Also crypto miners run everything balls to the walls to make as much cash as possible, when a PSU goes pop everything is vulnerable, a 100mhz gain on the core clock speed is irrelevant when you kill the gpu and have to shell out for a replacement
Last edited by Nathan_P on Sun Jul 30, 2017 8:27 pm, edited 2 times in total.
Nathan_P
 
Posts: 1400
Joined: Wed Apr 01, 2009 9:22 pm
Location: Jersey, Channel islands

Re: Please advise, hoping to build a dedicated folding machi

Postby Nathan_P » Sun Jul 30, 2017 8:12 pm

Aurum wrote:
ollie1983 wrote:...looking to build or buy a fairly brutish PC to fold with. It might get some VR gaming...
Hi Ollie, I don't game but I think there's a dichotomy in approaches. I think gaming benefits from a fast CPU RAM combination making an i7 look great or a Ryzen (more bang for the buck) look great, but these only have 24 PCIe lanes. Folding benefits from more cores (Xeon E5 with 40 PCIe lanes) and especially more PCIe lanes, not so much on RAM, but is often PCIe throughput limited. The best info I can recall to get into the nitty gritty is the thread foldy posted a couple back on PCIe bandwidth.
AMD MBs run all PCIe lanes through their Northbridge switch and this slows it down. Most Intel MBs route at least 8 lanes thru Northbridge. The only MBs I've found that route all 40 lanes directly from the CPU to the graphics card slots are the Gigabyte GA-X99 boards. Take a look at the overview web page and especially in the manual and find the block diagram of the MB. Where this matters is when you use four 1080 Ti's you can get x8x8x16x8 or with 3 cards x16x8x16 or x16x16x8.
I'm watching to see how MB makers implement the AMD Thread Ripper with 64 3.0 PCIe lanes. Sadly they always design these MBs for gamers and not scientists so no great expectations. I'd like to have either x16x16x16x16 or x8x8x8x8x8 directly from CPU to graphics cards and fewer bells 'n whistles and blinky lights.
http://wccftech.com/amd-ryzen-threadripper-1950x-cpu-performance-benchmarks-leak/


Expect at least x16x16x8x8 for GPU's on threadripper. the bigger problem is slot spacing. I would like to see high end boards take advantage of the 8-9 slot spacing you now get in high end cases to let the cards breathe better
Nathan_P
 
Posts: 1400
Joined: Wed Apr 01, 2009 9:22 pm
Location: Jersey, Channel islands

Re: Guidance for building a folding dedicated computer

Postby Aurum » Mon Jul 31, 2017 2:35 am

Hi Nathan, I was responding to a comment about building 5 to 7 card rigs not 3 to 4.
But those M.2 SSDs are so cute and fast and get rid of a brick and two SATA cables.
I don't put my rigs in coffins so a more spacious motherboard would be great. Sadly MB designers work for gamers and couldn't care less what's good for science.
User avatar
Aurum
 
Posts: 281
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: Guidance for building a folding dedicated computer

Postby ollie1983 » Mon Jul 31, 2017 5:11 pm

Hello again guys

What would you do, so far I have penciled a build in for about £3000

Haswell-E X99, 6850K under a Corsair 240mm liquid cooler.

16GB of DDR4 RAM from Kingston

The big I am not sure about is the GPU department.

Would you recommend 2 x GTX 1080 or a single GTX 1080ti by itself?
ollie1983
 
Posts: 7
Joined: Sat Jul 29, 2017 8:12 pm

Re: Guidance for building a folding dedicated computer

Postby Aurum » Mon Jul 31, 2017 5:32 pm

ollie1983 wrote:Hello again guys. What would you do, so far I have penciled a build in for about £3000
Haswell-E X99, 6850K under a Corsair 240mm liquid cooler.
16GB of DDR4 RAM from Kingston
The big I am not sure about is the GPU department.
Would you recommend 2 x GTX 1080 or a single GTX 1080ti by itself?
Definitely go with the 1080 Ti, I use EVGA 11G-P4-6393 and can fit 4 of them directly on the MB. The 1080 Ti has better energy efficiency 5.2 PPD/Watt versus 4.7 for a 1080.
The price of that CPU has come way down recently.
Is that 16 GB RAM a dual-kit? I've read they're faster.
What MB are you looking at?
You've got the makings of a fast rig.
User avatar
Aurum
 
Posts: 281
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: Guidance for building a folding dedicated computer

Postby Aurum » Mon Jul 31, 2017 5:38 pm

I see they're taking pre-orders for Threadrippers and X399 MBs. Way too overpriced for me. The MB only uses 48 of 64 lanes for graphics cards (x16x8x16x8), sad, so sad. TR looks like a major disappointment for folding. I'll stick with my GA-X99 plus Xeon E5 approach, $500 versus $1,400.
http://www.ebay.com/itm/Intel-Xeon-E5-2675-v3-OEM-CPU-1-8GHz-16-Core-SR1XM-Close-to-E5-2698v3-E5-2683-v4-/352009339940
https://www.newegg.com/Product/Product.aspx?Item=N82E16813128772R
User avatar
Aurum
 
Posts: 281
Joined: Sat Oct 03, 2015 3:15 pm
Location: The Great Basin

Re: Guidance for building a folding dedicated computer

Postby bruce » Mon Jul 31, 2017 5:48 pm

GTX 1080
GFLOPS: 8228 (8873)
TDP: 180
Price: $600

2 x GTX 1080

GFLOPS: 16456
TDP: 360
Price: $1200
GFLOPS/Watt: 46
GFLOPS/$: 13.7

GTX 1080ti
GFLOPS: 10609 (11340)
TDP: 250
Price: $700
GFLOPS/Watt: 42
GFLOPS/$: 15

I'd get the Ti and consider a second Ti when I can afford it.
bruce
 
Posts: 21326
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

PreviousNext

Return to FAH Hardware

Who is online

Users browsing this forum: No registered users and 1 guest

cron