RTX Super cards.

[gordonbb]

I gave up on AMD releasing working OpenCL drivers for the RX5700(XT) and picked up some 2060 and 2070 Supers instead. Initial results are in line with what others have reported so far but I’m still waiting on one shipment then after I rearrange the cards in my rigs to balance power loading I’ll start profiling them.

Getting the specific 2070s I wanted required keeping an eye on etailers as they’re still in short supply but the 2060 I found at a bricks and mortar store and are more obtainable on-line.

[MeeLee]

I was eyeballing a 2080 Super, and wondering if anyone has been able to test them out yet?

At $720 they're very competitive with my current favorite, the 2060.
Once you hit thermal or wall socket power limitations, a higher performing card becomes more important than a cheaper card.

If the 2080 Super cards can get close to 2M PPD in Linux, they would be doing nearly double what a 2060 can do, at also double the cost; and are pretty much even there!
Though the 2080 Super's power consumption will be much lower (estimated 185-190 Watt power capped, VS 125Watts for a 2060, or 250 Watts for 2 of them); also using up only 1 slot instead of 2 (this means, freeing up 1 CPU core, which leads to additional power savings).


With current prices, I think the most efficient folding rigs, are those that come with a motherboard with 3x full size PCIE slots (8x/4x/4x configuration), running a Core i3 9100 (4 cores, no GPU), 2x4GB of 2666Mhz DDR4 memory, and 3x RTX 2080 Super cards, with either a 2060 Super, 2070, or a 2070 Super in the 1x slot.
Bitcoin mining motherboards with multiple 1x slots don't really benefit for folding.
It's more efficient to run fewer, more powerful cards per motherboard, than more GPUs, all being bandwidth constricted.

Also, most motherboards nowadays only allow for 3 GPUs at a 8/4/4x speed configuration on their board's full size PCIE slots (all the Intel boards I tested that had 3x full size PCIE slots); and won't allow any additional slots on the M.2 adapters (save for a single 1x speed slot on one of the 1x slots) .
I've tested out many (about 8 intel) motherboards of different manufacturers and types, and no boards offered a '4/4/4x + more than 1x 1x slot'-configuration.
Even if the boards were equipped with 3 full size slots, and an additional 4x 1x slots, 3 of them will be disabled; and using them will just give a bios boot error saying more PCIE lanes than possible are used, and some cards were turned off...

If the board was equipped with 3x full size slot graphics cards, it would only allow access to 1 more 1x slot. That's it!


So populating the 3x full size slots with the most economic card, would be with 3x 2080 Super GPUs.



The remaining 1x (3.0) slot, preferably is equipped with a GPU no greater than an RTX 2070 / RTX 2070 Super, because the 1x speed slot will start to limit the performance of any faster card. (This all is under Linux btw)
On the other hand, the RTX 2060, 2060 Super, and 2070 all can run at >95% of performance at a mere 125 Watts, which is the lowest power cap these cards allow.
If Nvidia had allowed for lower power caps on the 2060, the 2060 would have been the best card to choose for this slot.
But since the 2070 gets better performance, at the same wattage, I'm thinking that a 2070 or 2070 Super would make for the most efficient card in a 3.0 1x slot in Linux.

Then again, the 2070 or 2070 Super is sold for quite a bit more than a 2060.
So it'd be a toss up between the 2 cards. (or just equip the remaining 1x slot, with any <2070 Super GPU you still have laying around).

[Shirty]

For information, in case anyone's interested. Windows clients:

[foldy]

RTX 2080 Super should be 1600k PPD average

[MeeLee]

@Foldy, 1.8-1.9M PPD according to my calculations.
Their cores are faster than older (non-super) RTX cards.

[Shirty]

As core counts increase, variance increases as well. The 2080Ti would easily average 2.5-2.7M ppd if it was only ever fed 150,000+ atom work units. It's crippled by less complex projects much more than mid-range GPUs.

I'm thinking about adding a few new cards to the roster possibly selling off the 1080Ti and mid-range RTX cards to help fund it. I'd be interested to see any 2080 Super results.

[MeeLee]

@Foldy, you're right. I was a little too optimistic, and made an error.
It's indeed closer to 1.6M PPD than the 1.8 I hoped it would.

Although, say you want to upgrade your server, and want to stay below a certain amounts of wattage; at which you're currently close to running to.
Which card would currently be the best upgrade from a 2060?

2070, 2070 Super, 2080, or 2080 Super?
Each one have only very little difference between them in performance.

[gordonbb]

I’m observing a 1770 kPPD average over 12 days for a 2080 Super with the behaviour typical of the 2080Ti in that they will drop dramatically in yield if fed a WU with a low atom count which these days is about half the time.
The 2070 Supers average 1543, 2060 Super 1370, 2070 1440 and a 2060 1030 kPPD over the same period.

My initial thoughts are the 2060 Super is the value king (PPD/$) if your just looking at individual cards but taking into account the whole system cost the 2070 Supers are more power efficient (PPD/W) and likely a better choice.

In comparison the 2070 Super has a smaller Standard Deviation than the 2080 Super and is a better fit for current WUs.

[bruce]

Good thinking, gordonbb.

We have attempted to assign larger WUs to big GPUs (including the 2080*) but that's not an easy process without some major changes to the Assignment logic. There are a lot of active GPUs with large numbers of shaders and here are a lot of smaller projects available to be assigned that need to be completed.

One alternative -- which could be added to the assignment logic, though it's clearly not a good choice -- is to have the AS test IF SHADER_COUNT>K1 AND ATOMS < K1, issue a message "No WUs available for your config." Personally, it think it's better to give you an assignment that's less efficient than you like than to leave your GPU idle. (I'm open to other ideas ... before I make a formal enhancement suggestion.)

[foldinghomealone2]

I think that there should be some kind of assignment logic.
Following rules could be applied:
1. Don't apply fast/big WUs to slow GPUs --> this increases the probability for fast WU/ fast GPU combination
2. Apply fast/big WUs to fast GPUs (like u described before).

In case of failed assignments, assignment should be repeated (eg. every 10secs) however under following conditions:
1. Never apply fast WU to slow GPU
2. Apply any WU to fast GPU after eg. 5 failed assignments

[bruce]

1. Don't apply fast/big WUs to slow GPUs --> this increases the probability for fast WU/ fast GPU combination
2. Apply fast/big WUs to fast GPUs (like u described before).

In case of failed assignments, assignment should be repeated (eg. every 10secs) however under following conditions:
1. Never apply fast WU to slow GPU
2. Apply any WU to fast GPU after eg. 5 failed assignments

How does the server determine if a WU is "fast"? Take a look at psummary and determine which projects should be/should not be assigned to slow GPUs/fast GPUs. (No fair looking at past history.)

[foldinghomealone2]

fast=more PPD
--> according to your own definition it would be 'many atoms'

[foldy]

Could even be more levels of atoms count/GPU speed.

Low atom count WU get category 1 and match slow GPUs category 1.
Middle atom count WU get category 2 and match middle GPUs category 2.
High atom count WU get category 3 and match fast GPUs category 3.

In future even more atom count WU and even faster GPUs get category 4 ...

If there is no WU for a category available then GPU gets also lower or higher category WU. But there must be a limit to not assign too big WU for a too slow GPU or else it cannot finish in time.

WU can be added to a category by atom count.
Or the GPU with limited PPD defines the category.

GPUs can be added to a category by shader count * Ghz.
Or the PPD can be taken for a GPU and that gives the category.

[bruce]

So all those categories are fluid ... category 1 or 2 can move uo or category 2 or 3 can move down any time projects are not getting their fair share of Donor Resources. FACT: FAH is about science, and if science is not getting done, it can easily be required to migrate resources to get the job done.

I don't see how any of this can actually be implemented without creating a huge historic database that has to be consulted to make any assignment. I want to establish categories that are pre-defined but ultimately flexible when things get too distorted. Nobody responded to my request to digest the data from psummary. If it's too hard for an intelligent human, it's not something that can be converted to useful code.

[gordonbb]

My feeling is that researchers being researchers, and nature abhorring a vacuum, will sooner rather than later catch up and release some projects with larger atom sizes to fill this void. I would rather have a card busy but under-utilized and doing useful work.

Perhaps bringing back a client-side preference for big WUs like in the SMP CPU folding days with a fall-back to normal if none are available.

Another option might be to be able to run multiple WUs on a single GPU but that could impede finishing generations within projects and likely would be way too much work to code and/or may not be possible with the libraries in use.