Hot Loads !!!

If you think it might be a driver problem, see viewforum.php?f=79

Moderators: slegrand, Site Moderators, PandeGroup

Re: Hot Loads !!!

Postby Spongebob25 » Wed Feb 22, 2012 8:55 pm

These WUs are a major problem. I have to 480 GTXs and had to majorly under clock them. Even with the under clocks, the cards where running at 90C with 100% fan. My case is well vented and I have another fan blowing air into the intakes of these cards. I hope this is not the future of GPU folding. Otherwise, I will stop folding on my GPUs and only fold on my CPU.
Image
Spongebob25
 
Posts: 113
Joined: Thu Jul 23, 2009 8:27 pm

Re: Hot Loads !!!

Postby BrainEater » Wed Feb 22, 2012 8:59 pm

I'm not experiencing any lag with the 803x units.

In fact I like 'em.They seem to run at 95-100% of the rated capacity for a 580 , aka 275-300 watts per card , with a 7.5kPPD increase per card..

....And I do want FaH to use my stuff to the fullest potential.

---------

However , I assume you guys actually test these before you send em out , and knew about the heat/power increase.

My UPS would have asked for some warning.In my case , I woke up to UPS overload beeps because of the 200 Watt jump.....If I was running a 1-3-5 tri card setup I'd have woken up to the smoke detector.

:lol:

I will say again tho , I do like these units.Good work.
BrainEater
 
Posts: 17
Joined: Sat Mar 28, 2009 3:16 pm

Re: Hot Loads !!!

Postby Uniwarking » Wed Feb 22, 2012 10:05 pm

The very fact that this thread exists, as well as others, indicates that something has changed sifnificantly. In making my posts, I wasn't saying "change it back or else," I was simply stating that I wasn't willing to fold with my GPU at 90C + with a fan at 90%+.

I'd be happy to continue folding, but if actually using my computer for what little time I do get to use it becomes an issue... I don't see how I or others can continue. The pause mode doesn't even work on the GPU client... thats another gripe of mine.

In order for folding to become more widespread, the user interface, setup, and user experice really need to be improved. Just my .02...
Uniwarking
 
Posts: 5
Joined: Tue Jan 31, 2012 10:48 pm

Re: Hot Loads !!!

Postby Jesse_V » Wed Feb 22, 2012 10:20 pm

Uniwarking wrote:The pause mode doesn't even work on the GPU client... thats another gripe of mine.

In order for folding to become more widespread, the user interface, setup, and user experice really need to be improved. Just my .02...


Your two cents is worth something. The PG is working on v7, the next generation of F@h software that has all of those things and more.
http://folding.stanford.edu/English/WinGuide
https://fah-web.stanford.edu/projects/FAHClient

I don't know enough to say whether it will help your heat issue one way or the other, but it certainly has better usability.
Pen tester at Cigital/Synopsys
User avatar
Jesse_V
 
Posts: 2773
Joined: Mon Jul 18, 2011 4:44 am
Location: USA

Re: Hot Loads !!!

Postby k1wi » Wed Feb 22, 2012 10:24 pm

Unfortunately, I believe the issues expressed here are only going to get worse in the future (and I don't believe it is FAH's fault).

In my opinion the fault largely lies with the way hardware is being designed. First, GPUs are being designed to pretty much only use a limited amount of their theoretical performance and second, because of the stratification of GPUs, different WUs have a different impact on performance (relative to the benchmark machine).

In the first instance, GPU makers are increasingly looking at their hardware and noting that most of the time, even when they are being used, GPU resources aren't actually being utilised anywhere near 100%. As a result, they are able to release products that are very stable at speeds/performance attributed to every day use, such as gaming, but which actually are not up to muster when they are being used by applications at or very near to their peak performance. This is why applications such as FurMark (which they ironically call power viruses) and some games that are better at utilising GPUs are being actively throttled by different manufacturers (OCP for NVidia and Powertune for AMD). FAH can't do anything about that, although it would be interesting to see whether anyone has had their GPU downclocked as automatically a result OCP/powertune?

In the second instance, there is possibly a little bit that FAH can do here, but at best I can see it being rather labour intensive and potentially futile. WUs utilising GPUs with large resources more effectively than other WUs that utilise the smaller ones more effectively, result in, amongst other things, PPD that does not follow the benchmark. This was seen when the lower rated 400 & 500 series fermis were producing disproportionately high PPD relative to the 480s on the smaller WUs (because the resources on the more powerful 480s weren't being fully utilised). From a productivity perspective, it would be ideal for WUs to be sent to the hardware that folds it the most efficiently, so smaller WUs would be sent to smaller GPUs and larger WUs would be sent to the larger GPUs. However, to do so would be pretty damn time consuming. Just look at the effort that is required to keep a whitelist and imagine multiplying that by each project grouping!
k1wi
 
Posts: 1152
Joined: Tue Sep 22, 2009 10:48 pm

Re: Hot Loads !!!

Postby 7im » Wed Feb 22, 2012 11:00 pm

BrainEater wrote:...

However , I assume you guys actually test these before you send em out , and knew about the heat/power increase.


Yes, they do get tested, by the beta team. And yes, they did know. Non beta team members should still have read only access to observe the comments of those who test new projects when you log in to the forum. A few beta team members commented on the higher power draw and slightly better PPD over there.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
User avatar
7im
 
Posts: 14648
Joined: Thu Nov 29, 2007 4:30 pm
Location: Arizona

Re: Hot Loads !!!

Postby k1wi » Wed Feb 22, 2012 11:27 pm

7im wrote:
BrainEater wrote:...

However , I assume you guys actually test these before you send em out , and knew about the heat/power increase.


Yes, they do get tested, by the beta team. And yes, they did know. Non beta team members should still have read only access to observe the comments of those who test new projects when you log in to the forum. A few beta team members commented on the higher power draw and slightly better PPD over there.
+1. It's not required for regular members to follow the read-only forum, but it can be of benefit if you do.
k1wi
 
Posts: 1152
Joined: Tue Sep 22, 2009 10:48 pm

Re: Hot Loads !!!

Postby codysluder » Wed Feb 22, 2012 11:57 pm

k1wi wrote:Unfortunately, I believe the issues expressed here are only going to get worse in the future (and I don't believe it is FAH's fault).

In my opinion the fault largely lies with the way hardware is being designed. First, GPUs are being designed to pretty much only use a limited amount of their theoretical performance and second, because of the stratification of GPUs, different WUs have a different impact on performance (relative to the benchmark machine).

In the first instance, GPU makers are increasingly looking at their hardware and noting that most of the time, even when they are being used, GPU resources aren't actually being utilised anywhere near 100%. As a result, they are able to release products that are very stable at speeds/performance attributed to every day use, such as gaming, but which actually are not up to muster when they are being used by applications at or very near to their peak performance. This is why applications such as FurMark (which they ironically call power viruses) and some games that are better at utilising GPUs are being actively throttled by different manufacturers (OCP for NVidia and Powertune for AMD). FAH can't do anything about that, although it would be interesting to see whether anyone has had their GPU downclocked as automatically a result OCP/powertune?

In the second instance, there is possibly a little bit that FAH can do here, but at best I can see it being rather labour intensive and potentially futile. WUs utilising GPUs with large resources more effectively than other WUs that utilise the smaller ones more effectively, result in, amongst other things, PPD that does not follow the benchmark. This was seen when the lower rated 400 & 500 series fermis were producing disproportionately high PPD relative to the 480s on the smaller WUs (because the resources on the more powerful 480s weren't being fully utilised). From a productivity perspective, it would be ideal for WUs to be sent to the hardware that folds it the most efficiently, so smaller WUs would be sent to smaller GPUs and larger WUs would be sent to the larger GPUs. However, to do so would be pretty damn time consuming. Just look at the effort that is required to keep a whitelist and imagine multiplying that by each project grouping!


+1

Would you buy a car if the radiator boiled over every time you got up to freeway speeds? Just cars spend most of their time on city streets or in heavy traffic doesn't make it right to design them with a tiny radiator and expect that anybody who expected to make a cross-country drive to either install an oversized radiator or install a governor that limits you to 90% of freeway speed?
codysluder
 
Posts: 2128
Joined: Sun Dec 02, 2007 12:43 pm

Re: Hot Loads !!!

Postby MtM » Thu Feb 23, 2012 12:52 am

None of the people of the team I fold for have lag problems ( edit: spoke to soon here :lol: ), but power draw did increase ( the same ups stories come up ).

The same question was raised, what if it had been a multi gpu setup with 4 cards ( like many ran in the days the gpu's where the most point heavy )? Haven't heard from someone who ran into that, no one has more then 2 fermi's running.

Also to support the above comments about build quality, the quadro equivalents of any geforce release all have substantial lower clocks ( and therefore can be warranted to run gpgpu tasks ). A consumer class gpu does not have the same quality power circuitry, and runs higher clocks on reference cooling solutions which are aimed to run consumer gpu tasks. This includes the occasional gpgpu assisted video conversion, but I don't believe manufacturers expect their consumer cards to be used for gpgpu 24/7x365. Nor do I think they should.

The load is not consistent with 'normal' consumer usage, and you don't build products to have features your normal consumer base does not need, not without being not competitive because of increased prices. You don't get warranty on your four wheel drive sedan if you take it to to Amazone.

If you use your card for something which is not consistent with everyday use, you prepare it by using aftermarket ( cooling ) solutions, just as you would add roll bars to any off road vehicle which doesn't have them before going out in the bush.

Sorry for the car analogies, they are to easy to ignore.

Also, to people who ask about testing, I want to +1 the advice to read the beta team section, and better yet ask for admittance and help catch any issues like this as soon as possible. Maybe if a bigger group during beta team voiced their concerns, the release of these to general f@h had been either moved back or had been accompanied with a warning in the general forum about what people could expect. It's a good thing if you have concerns to be able to discuss them at a time when the most effect can be achieved.

Disclaimer, I'm not saying such a scenario would take place or should have taken place, just trying to emphasize the importance of beta team to everyone who has not before considered joining up.
MtM
 
Posts: 3054
Joined: Fri Jun 27, 2008 2:20 pm
Location: The Netherlands

Re: Hot Loads !!!

Postby k1wi » Thu Feb 23, 2012 1:19 am

I really don't consider running GPGPU 24/7 to be abnormal behaviour... After all, consumer GPUs are being increasingly designed for compute tasks (take AMD's latest architecture). If you design a GPU to have a peak theoretical performance of 1TFLOP, should it not be designed to be able to run at such a performance indefinitely? You would expect your CPU to, I would expect my CPU to. If it did not because it was overclocked (and stable for smaller projects), then the overclock should be pruned back.

How often do manufacturers quote peak performance and how frequently do they say "but you will never utilise it all because your hardware will be throttled before you get there"?

As to this needing to have been notified... At what point should a work unit be notified? 1 degree above the temperature experienced folding small work units, 5 degrees? The temperature difference between WUs is dependent on the relative size of WU... Which is a descrete scale rather than a small/big scale.
k1wi
 
Posts: 1152
Joined: Tue Sep 22, 2009 10:48 pm

Re: Hot Loads !!!

Postby MtM » Thu Feb 23, 2012 1:47 am

1. I'm taking the devils stance, I'm closer to your standpoint but I do know that if you really want what you're asking for you'll have to pay for it on the counter. Again, notice pcb differences between quadro and geforce cards, and add lower clocks on the quadro cards. Do you think manufacturers should sell lower clocked cards so they can run gpgpu tasks 24/7, and offer lower performance for gamers who don't run their cards at 100% all day? Or do you think gamers should pay more for cards with the same clocks which have additional cooling and power circuitry to match those used on professional gpgpu designated add in boards?

2. Not sure, but doesn't hurt to throw the idea up in the air so it can be discussed ( and dismissed if deemed not in order ). And I repeat, this might be a good chance to signify the importance of people's participation in beta testing, my post isn't focused on this particular work unit alone.
MtM
 
Posts: 3054
Joined: Fri Jun 27, 2008 2:20 pm
Location: The Netherlands

Re: Hot Loads !!!

Postby mdk777 » Thu Feb 23, 2012 3:04 am

well, I seem to remember this kind of debate in the past.

However, at that time all GPU FOLDING was considered pure BETA so the heat and wattage issues were considered par for the course.

With GPU FOLDING going mainstream with the V7 client, I think PG may have to think these ramifications through in more detail.

Just my 2 cents. :mrgreen:
Transparency and Accountability, the necessary foundation of any great endeavor!
mdk777
 
Posts: 813
Joined: Fri Dec 21, 2007 4:12 am

Re: Hot Loads !!!

Postby GreyWhiskers » Thu Feb 23, 2012 8:24 am

GreyWhiskers wrote:The 762x projects were, and still are, kept as Advmethods after a similar hue and cry when they came out. That gave the GPU folders some control over their destiny.


I was too subtle here. What I was suggesting is that the new 803x WUs be kept as ADVANCED, as is still the case for the very stressful 762x projects. On my desktop with the GTX560Ti, over which I have a lot of control for cooling and clock, I will keep the ADVANCED flags set, via setting as BETA, and get the 762x or 803x. I've been running like this for almost a year, and it is working well and productively.

But, I prefer not for my Fermi GTX560M laptop. The laptop GPU has a default clock of 775 MHz. I have been running it underclocked at 725 MHz for the "normal" non-ADVANCED projects. When processing an 8033, the GPU temp, as reported by Afterburner, was running 92-93 deg C, vs 70 deg C for others. I tried further underclocking to 650 MHz, and the temp hardly budged - down to maybe 90 deg c.

Is it a capacity issue? Are there not enough GPU clients, either V6 or V7, configured to accept ADVANCED WUs to complete the 803x projects on their timeline, and the general FAH Fermi GPU pool needs to be tapped? Or is it feasible to leave the 803x on ADVANCED?
User avatar
GreyWhiskers
 
Posts: 767
Joined: Mon Oct 25, 2010 5:57 am
Location: Saratoga, California USA

Re: Hot Loads !!!

Postby 7im » Thu Feb 23, 2012 5:45 pm

Anyone tried reducing the CPU usage setting to starve the GPU of a little data, introducing wait cycles, reducing lag and heat? Yes, the trick isn't perfect, and doesn't work for everyone, but can help some...
User avatar
7im
 
Posts: 14648
Joined: Thu Nov 29, 2007 4:30 pm
Location: Arizona

Re: Hot Loads !!!

Postby MtM » Thu Feb 23, 2012 6:16 pm

7im wrote:Anyone tried reducing the CPU usage setting to starve the GPU of a little data, introducing wait cycles, reducing lag and heat? Yes, the trick isn't perfect, and doesn't work for everyone, but can help some...


viewtopic.php?p=208938#p208938

I agree, and feel stupid for not looking there sooner. Must have been because I had not noticed the -cpu x value being passed to the fahcore and it actually working. Bad memory or not paying attention :oops:
MtM
 
Posts: 3054
Joined: Fri Jun 27, 2008 2:20 pm
Location: The Netherlands

PreviousNext

Return to V6 GPU3 beta (including Fermi) OpenMM

Who is online

Users browsing this forum: No registered users and 1 guest

cron