Atlas Folder Update - 19 GTX295s

[Atlas Folder]

Hey everyone! Atlas Folder, my GPU folding farm has finally reached its initial build-out goal!

I know a few of you have been keeping up with my build so I thought I've give you a head's up to have a look:

[Atlas Folding]



Jason

[anandhanju]

Atlas Folding reeks of Awesomeness! Congratulations on the successful build

[toTOW]

Impressive !

[kiore]

respect!
kiore.

[COOLDUDEGAMER]

All I have to say is ... WOW! That is just plain (and now I have a word for this) Awesomerest. A new word! In other words that is one great setup.

[redshift]

Very nice! I hope your efforts go along way in the research for curing Huntington's.

Now, if only I could put one of those in my house (without rewiring)....

[Van_Arnam]

most impressive (and tidy!). everyone note the ps3 up top

how hot does that room get?

[alpha754293]

Have you considered running liquid cooling for the entire system in order to improve your systems engineering management? Just another thought.

I think that if you were to have all of the systems up and running, you expecting anywhere between 240-270k PPD right?

[shdbcamping]

That is absolutely RAD. Great Job!

[MESeidel]

Well done Atlasfolding.
And keep on posting updates on your Blog, very interesting reads ;o)

Have you considered running liquid cooling for the entire system in order to improve your systems engineering management? Just another thought.

I think that if you were to have all of the systems up and running, you expecting anywhere between 240-270k PPD right?

Water blocks for the GTX-295 cost like 1/3 of the Cards price.
Plus the Radiator and other Stuff.
The increase of OC potential is not so big, on the other hand.
Buying another 4 Cards, Mainboard and the power to run it should be cheaper for very long time.

Only if the noise is killing one (like me^^), Water cooling should be interesting for such a Server System.
The price is the reason why I stopped thinking about the GTX-295 being an option for my own F@H-Garden update...

[Smirnoff]

Wow, it's a pretty awesome setup.

[dempaSD]

249k points yesterday I am impressed, to say the least. Was it one day without beta testing? And no overclocking? How do the GPU temps look?

Thanks for sharing for science!

[Atlas Folder]

Water cooling isn't practical. The expense, the risk to the cards of modification and voiding the warranties on $9,600.00 worth of video cards is not something that I can justify. It also makes ebaying them one day to upgrade to 395s more difficult. I can see it now... For Sale: 19 water cooled GTX295s, never used for gaming.

The GTX295s have proven to be a bit tweaky en masse and I haven't found the cause/solution yet. I have one, sometimes two, GPUs per machine in EUE at any given time. That's one reason the rig has not hit its maximum potential yet. I don't believe its temperatures, the cards all run with fans at 100% and the temps sit at 72 degrees C or lower across the board. One machine seems to have a motherboard problem for instance -- one of the GPUs in any card I put in that slot will not function. I've changed power supply cabling as well. That particular half of any card I put in the slot doesn't work.

The heat output isn't *that* bad; it's like standing in front of a wall of low-wattage hair dryers. Or maybe 4 restroom hand-dryers... I'll shoot a little video of it tonight and post it somewhere because it *is* pretty loud.

No, I was not burning beta WUs yesterday so my PPD saw a huge jump.

Jason

[bruce]

VERY impressive.

The GTX295s have proven to be a bit tweaky en masse and I haven't found the cause/solution yet. I have one, sometimes two, GPUs per machine in EUE at any given time. That's one reason the rig has not hit its maximum potential yet.

Let's treat this as a new issue and see what data we can gather. There will always be a small percentage of WUs which have spontaneous errors. The project owner adjust some parameters during the design and beta testing of the project which keep the number of EUEs to a small fraction of the WUs but with 19 GPUs, that's 19 times a small fraction which may be approaching a high enough frequency for you to notice.

Please report the Project/Run/Clone/Gen numbers and the error messages and a Moderator can help track whether it seems to be a problem with the WU itself or with something on your end. Also, keep track of whether the EUEs favor one particular GPU or not.

Make enough posts in the Problems with a specfic WU forum that we can (hopefully) spot some trends.

[alpha754293]

Water cooling isn't practical. The expense, the risk to the cards of modification and voiding the warranties on $9,600.00 worth of video cards is not something that I can justify. It also makes ebaying them one day to upgrade to 395s more difficult. I can see it now... For Sale: 19 water cooled GTX295s, never used for gaming.

The GTX295s have proven to be a bit tweaky en masse and I haven't found the cause/solution yet. I have one, sometimes two, GPUs per machine in EUE at any given time. That's one reason the rig has not hit its maximum potential yet. I don't believe its temperatures, the cards all run with fans at 100% and the temps sit at 72 degrees C or lower across the board. One machine seems to have a motherboard problem for instance -- one of the GPUs in any card I put in that slot will not function. I've changed power supply cabling as well. That particular half of any card I put in the slot doesn't work.

The heat output isn't *that* bad; it's like standing in front of a wall of low-wattage hair dryers. Or maybe 4 restroom hand-dryers... I'll shoot a little video of it tonight and post it somewhere because it *is* pretty loud.

No, I was not burning beta WUs yesterday so my PPD saw a huge jump.

Jason

hehe...I'm used to Sun servers where it's typically 80-90 dBA. If I have my own company, I'd probably be using a turbine driven blower system in order to run it slightly more efficiently, and such that all of access points into the room will be controlled (naturally), and will have a sign "ear protection required beyond this point", possibly eye protection as well because of the turbine driven fan system. (The advantages of using a very, very small, micro-turbine from Capstone turbine for example. Probably not more than 30 kW, and looking for a drive shaft speed of at least 30 krpm.

The watercooling is just an idea because I was doing more research into the IBM POWER6 thermal management system and it gave me a few ideas in terms of a microchannel (well...a very wide microchannel albeit) cooling method that might work for you, and I think that if you pair up with someone who's used to do it for gaming systems; it might be something to consider. I don't know. Just an idea.

Personally, if it were me, I would have tested all of that in the test build that you had, before moving it over onto the rack. But that's just me, only cuz then I'd actually have a semi-practical use for my engineering degree. (*bored*). If I also did that, that would mean that I could theorectically be able to pack 3 of them into a 2U rack using PCIe x8 risers where the slots are also mechanically cut in order to fit the PCIe x16 cards.

How are you controlling/managing the systems? KVMoIP? or are you running completely headless and then you get access to the systems via ssh/telnet?