ATI and NVIDIA stats vs. PPD numbers

[P5-133XL]

Client stats seem to indicate that ATI GPU's and NVIDIA GPU's are doing a similar amount of work. Since, the all WU's are benchmarked against both brands why is NVIDIA getting much more PPD? The stats seem to be contradictory to the arguement that NVIDIA is so much faster than ATI and thus deserves more PPD.

OS Type Current TFLOPS* Active CPUs
ATI GPU 475 4316
NVIDIA GPU 1751 15922

From those numbers I get ATI is producing .11TFLOPS/client and NVidia is producing .1099TFLOPS/client. It has been my observation that NVidia gets over twice the PPD of an equivilent ATI card. These numbers seem to be contradictory.

So is it that there are a disproptionate number low-end NVidia dragging down its average stats while there are large proportion of high-end ATI cards so that the average Teraflops is not correlating with the benchmarking?

[DreadedOne509]

The stats are skewed. You'll need to look at the individual work units assigned to each of those GPU's
and the returns on them to get a more accurate picture.

[P5-133XL]

The stats are skewed. You'll need to look at the individual work units assigned to each of those GPU's
and the returns on them to get a more accurate picture.

What is the value in supplying stats that are skewed and don't communicate useful data?

I rather look at the individual WU's point values/PPD as instantanous values, while the TFLOPS as to be an long-term average. Like a car on a trip: You can look at the start time, end time, and miles travel and come up with an average miles per hour. While that does not indicate if at a particular point one is speeding, if does tell how fast you were going overall.

For machines with equivilent TFLOP averages, the overall PPD's should be similar: They are doing the same amount of work. When they aren't then there is a problem and that is all I'm trying to point out. Maybe the problem is the stats, and then that needs to be fixed. Maybe the problem is in calculating the PPD's and then that is the item that needs to be fixed.

So what specificly is wrong with the stats so that are producing the skewed results?

[MtM]

Client stats seem to indicate that ATI GPU's and NVIDIA GPU's are doing a similar amount of work. Since, the all WU's are benchmarked against both brands why is NVIDIA getting much more PPD? The stats seem to be contradictory to the arguement that NVIDIA is so much faster than ATI and thus deserves more PPD.

OS Type Current TFLOPS* Active CPUs
ATI GPU 475 4316
NVIDIA GPU 1751 15922

From those numbers I get ATI is producing .11TFLOPS/client and NVidia is producing .1099TFLOPS/client. It has been my observation that NVidia gets over twice the PPD of an equivilent ATI card. These numbers seem to be contradictory.

So is it that there are a disproptionate number low-end NVidia dragging down its average stats while there are large proportion of high-end ATI cards so that the average Teraflops is not correlating with the benchmarking?

Why do you assume Tflop is equivalent to scientific value? PPD should represent that, not maximum floating point operations per second which is btw only a synthetic number.

Why do you think a card which get's a certain tflop rating from it's vendor, will reach that potential utilising code bases still in early beta stages?

And most importantly, why are the results skewed? Their not, considering the current benchmark system.

If you ask if it's a fair system, no I don't think so. But since it's in place, there are no skewed results. You complain Nvidia get's to much points? Want to come to xs and tell the team there that? You'll get lynched because we just had the 5748 bomb dropped on us. Ppd plummited and people where screaming bloody murder, ready to get the nooses and lynch someone. But, still, with the current point system, the ppd is in line with what most people knew. Nvidia got the better hand with the smaller wu's, but when the atom size will increase, Nvidia will take the bigger hit compared to Ati. It's not screwed results at all, it's results due to the benchmarking machine being an Ati card.

[P5-133XL]

I assume TFlops is equivalent to scientific value because it is a measure of how much floating point calculating is being done. More calculating, the more scientific work being done.

your description of Tflop rating is not what the stats measure. It is not a rating from the vendor according to the stat page.

*TFLOPS is the actual teraflops from the software cores, not the peak values from CPU/GPU/PS3 specs.

I am not saying the benchmarking system is fair or unfair. I'm saying there is an inconsistancy here: They should correlate to each other and they don't!

[MtM]

I assume TFlops is equivalent to scientific value because it is a measure of how much floating point calculating is being done. More calculating, the more scientific work being done.

your description of Tflop rating is not what the stats measure. It is not a rating from the vendor according to the stat page.

*TFLOPS is the actual teraflops from the software cores, not the peak values from CPU/GPU/PS3 specs.

I am not saying the benchmarking system is fair or unfair. I'm saying there is an inconsistancy here: They should correlate to each other and they don't!

Good correction I was to quick, iirc the tflop rating is done by the client cores before requesting work from the servers, forgot about that for a moment.

[7im]

I assume TFlops is equivalent to scientific value because it is a measure of how much floating point calculating is being done. More calculating, the more scientific work being done.

your description of Tflop rating is not what the stats measure. It is not a rating from the vendor according to the stat page.

*TFLOPS is the actual teraflops from the software cores, not the peak values from CPU/GPU/PS3 specs.

I am not saying the benchmarking system is fair or unfair. I'm saying there is an inconsistancy here: They should correlate to each other and they don't!

This has been discussed already. Please search for it.

No, TFLOPS to do not equal scientific value in a linear manner for a number of reasons, already explained in that other thread. TFLOPS is not a good comparison tool between clients. And no, they should not correlate as you might expect.

Here is one example: http://foldingforum.org/viewtopic.php?f=50&t=5632
Another: http://foldingforum.org/viewtopic.php?f=50&t=4863
See also Vijay's quote on the matter, search for the term "deceptive"

If no objections, I think we can close this.

[MtM]

Please do

[P5-133XL]

We'll start out with the fact that the deceptive term has to do with comparing clients that do folding two different ways: CPU vs GPU -- Where there were more variables, such flexibility of the type of work and different deadlines. My observation does not contain those variables. I am comparing like vs. like when comparing GPU2 client vs GPU2 client . I stand by my conclusion that there is an inconsistancy here and strongly disagree, that there is any deceptive variables, since the same WU's are benchmarked on both clients and so there should be a correlation.

Next, the first link you supplied does not contain anything to do with TFlops and their association to Benchmarking that I can see. It had to do with the vagarities of the Benchmarking system. I'm not complaining that there is a problem with the benchmarking system. I'm not saying there is a problem with the client stat system. I'm saying that there is a problem somewhere because they don't correlate and they should. I have no idea where the problem is. So quit trying to close down this discussion based on the fact that there are lots of discussions on the Benchmarking system

As far as I can tell, this subject has never been discussed before. Yes, I remember the GPU vs CPU TFLOPS discussion and the arguement was that GPU TFLOPS were inheirently different than CPU TFLOPS. I'm willing to accept that at face value. Now where is the inheirant deceptive difference between two GPU2 clients that run the same WU's when being benchmarked with no significant deadline differences? Now when someone successfully addresses the actual issue, then I would have no problem with closing down the thread. Till, then I object because as far as I can tell. you are just playing a game trying to silence a legitimate observation and question by trying to label it as something it is not.

[Amaruk]

...since the same WU's are benchmarked on both clients and so there should be a correlation.

All WUs are benchmarked on ATI. Per the FAQ: How do you decide the credit value of GPU work units?

Points are determined by the performance of a given machine relative to a benchmark machine, similar to the CPU client benchmark process. Before releasing any new project (series of work units), we benchmark it on a dedicated computer with an ATI Radeon 3850 GPU (512 MB, 320 Stream Processors), running in a Dell Inspiron 531, with a 2.16 GHz dual core AMD 64 X2 4000+.


Hypothetically speaking, if the Nvidia cards are better (more efficient) than ATI at folding smaller proteins, one could expect a higher PPD in relation to TFlops.

As the size of the proteins increases, the Nvidia cards become less efficient compared to ATI. Since the benchmark is the ATI card, one could expect the PPD/TFlop for Nvidia cards to drop.

JMHO

[codysluder]

The stats are skewed. You'll need to look at the individual work units assigned to each of those GPU's
and the returns on them to get a more accurate picture.

Right.

If two cars are driven differently (mostly short trips in city traffic vs. longer trips on uncongested freeways) you would expect the actual MPG to differ from th manufacturer's report.

Nvidia has gotten mostly small WUs; ATI has gotten a broader mix of WUs.

Nvidia does very well on small WUs but produces EUEs on larger WUs so the larger WUs have been temporarily suspended for ATI until the problem can be resolved. As long as the "average" ATI WU ant the "average" nvidia WUs are different, the statistics well be skewed.

[P5-133XL]

So the Client stat algorithums need to be fixed so that they truely reflect the amount of calculating going on. Or the alternative is to discontinue their use because they don't actually represent what is going on: No data being better than wrong data.

[codysluder]

If a WU from one project takes X hours on the benchmark machine but can only use yy MFLOPS an another project also uses X hours but it uses zz MFLOPS because the software efficiency is different you're suggesting that the points should be based on yy and zz rather than on X. Who do you want to be responsible when your hardware is unable to use the rated MFLOPS (which is essentially all of the time)?

The benchmarks are based on actual time, not on theoretical throughput.

[P5-133XL]

ZZ and XX already include the time (The PS in MFLOPS stands for per second), so X hours has already been taken into account.

There is no time where the rated MFLOPS is not being used, because the client stats are not using manufacturer-inflated rated peak numbers but are using the actual TFLOPS of the client as it is being used. There is a difference between theoretical and actual and the client stats claim to be using actual TFLOPS.

So the question still remains -- What is wrong with the stat page since it is not correlating with the amount of work being done as reported by PPD's?

[codysluder]

Oh, you're suggesting that ACTUAL TFLOPs be used. So if one WU uses a few percent fewer actual FLoating point OPerations, you wont mind getting 97.89 points for a WU while somebody else gets 100.87 points for the same project? If all FahCores report total actual FLOP numbers, it could be done, but it wouldn't be good if any FahCore happens to do mostly integer OPs.