GPU CORE22 0.0.2 coming to FAH - p11737-9 feedback thread

[MeeLee]

Now that I officially enabled the 'beta' flag,
Here's some of my initial overclocking/power capping findings:

- First off, Core 21 or Core 22 run pretty stable at the same power cap and overclock levels.
The initial WU that just abruptly failed, may have been the oddball, I haven't seen it repeated; and most of the beta WUs do seem to resume from last checkpoint.

- Core 22 gets approximately 40% higher PPD score at the same power cap.
The difference however, is that core 22 does the same job at a much lower core frequency.
Meaning, if my GPU has a peak boost frequency of 1935Mhz, and with Core 21 I overclocked and capped the power to run at ~1905Mhz, core 22 would run the same settings with 40% higher PPD, at ~1650-1675Mhz.
Meaning, I need to somehow increase the GPU boost frequency.
Since my overclock (+135Mhz) on the GPU is already set to the maximum stable overclock under core 21, this can only be done, by increasing power to the GPU.
And while my 2080Tis ran pretty close to stock PPD speeds, at a mere 160-169W on Core 21,
Core 22 requires me to increase the wattage (my initial settings are set to 190-195W). Which is close to stock wattage.
I haven't yet had the time to play around with finetuning on core 22. But I'm close.
This in turn increases PPD rating from 40% to around 60%.

- Increasing the wattage per card, will cause the overclock settings to cause failed WUs.
Eg: if my 2080 has a 135Mhz overclock, and runs stable at 1870Mhz, with core 22, increasing the power levels to 195W gets me ~1815Mhz, but I will have to reduce the overclock by a few Mhz (usually 5 or 10Mhz).
This is common even on core 21. When power levels are increasing, overclocking levels need to decrease. When power caps are in effect (and maximum power draw is reduced by 50-100Watts), stable overclocking can be increased (by a 10-25Mhz amount).

- Running higher wattage settings will result in hotter running GPUs.
Core 22 running at 55-59C up from 45-50C on Core 21; with a ~60% increase in PPD in FAHControl.

- If power caps are too low (eg: 125W on Core 22), I believe I saw the 2080Ti run the GPU into passive mode (800Mhz).
That number I saw on core 21 at 110W.
I may be mistaken here, but it may indicate that when the power cap is set too low, you might actually see a LOWER PPD rating on core 22 than on core 21; which could be one explanation on why some people see a reduction in points compared to core 21.
But this is not confirmed yet.



My next project will be to look into if it's worth running 4GPU system at lower power levels, vs 3GPU system with all GPUs running at their stock power levels.
I'm still in the test phase on them and hope the beta WUs will run for a while longer.

[squads]

Is it to be expected that Core22 will require more PCI bandwidth? My 1060 that is on a 1x riser sees a drop in PPD with p11737 and it looks like the PCI bus is saturated (Nvidia control panel showing 80+% utilization). A 2060 and 1660 Ti in 8x slots on the same machine see a significant increase in PPD vs a typical Core21 WU.

[antropofob]

Is it to be expected that Core22 will require more PCI bandwidth? My 1060 that is on a 1x riser sees a drop in PPD with p11737 and it looks like the PCI bus is saturated (Nvidia control panel showing 80+% utilization). A 2060 and 1660 Ti in 8x slots on the same machine see a significant increase in PPD vs a typical Core21 WU.

On Windows or Linux?

[MeeLee]

Is it to be expected that Core22 will require more PCI bandwidth? My 1060 that is on a 1x riser sees a drop in PPD with p11737 and it looks like the PCI bus is saturated (Nvidia control panel showing 80+% utilization). A 2060 and 1660 Ti in 8x slots on the same machine see a significant increase in PPD vs a typical Core21 WU.

Yes, that is confirmed. Pcie 3.0 1x is a bottleneck for RTX GPUs. I'm sure pcie 2.0 1x will be a bottleneck for GTX GPUs.
Pcie 3.0 4x shows 50%bandwith utilization in Linux. Might be a bottleneck in Windows, not confirmed.

FAH is preparing and ready for pcie 4.0 1x configurations with core 22.

[foldy]

How many CPU threads are available for the GPU?

[MeeLee]

It appears one of my GPUs only gets core 21 work assigned, even with the beta flag enabled.
Is that normal?

[bruce]

Beta testing is a dynamic process. New projects and/or software are introduced, then tested for as long as it takes to isolate the problems and fix them. What you get today may change tomorrow. The developers are rolling out new projects gradually and potentially fixing things in FAHCore_22 over a period of several weeks. Yes, it's normal ... until it changes again.

Be patient and let the developers make assignment changes whenever it seems appropriate to their testing plan. There's never a guarantee that you'll get a beta assignment. Everything that completes beta testing will be moved to Advanced (unless it needs to be discontinued).

Please read and understand the policies involved in beta testing. An Overview Of F@h Beta Team

[squads]

How many CPU threads are available for the GPU?

12 thread CPU feeding 3 GPUs and otherwise doing nothing, so that is a non-factor

On Windows or Linux?

Linux

[BlazingDragon]

I have a couple of Radeon VIIs. They are dedicated to folding.

Results for the 1173x Projects shown below [from manually scraping the log files and manipulating via a spreadsheet pivot table].
** edited 2/2 with more results **

Code: Select all

Project	GPU	         Attached	       Average - PPD	StDev - PPD	Average - Duration	WU Count
11737	Radeon VII [A]	[Riser] PCIE x1	792,591.19	   8,773.21	   01:57:55	          11
	     Radeon VII [B]	Mboard PCIE3 x16	794,596.92	  25,253.20	  01:57:44	          13

11738	Radeon VII [A]	[Riser] PCIE x1	1,784,835.71	9,588.54	    02:49:53	          6
        Radeon VII [B]	[Riser] PCIE x1	1,918,760.26	118,672.47	  02:42:22	          4

11739	Radeon VII [A]	[Riser] PCIE x1	1,484,794.61	174,681.16	  01:41:23	          6
	     Radeon VII [B]	[Riser] PCIE x1	1,645,442.51	221,238.59	  01:34:42	          2

Details:
I originally had one in standard tower [Ryzen 7 1700 with 16GiB RAM], set up as the primary card [A], and the 2nd card in a 6U GPU case [Ryzen 5 1600, 8 GiB RAM], using a PCIE3-x1 riser card [as a secondary card]. The CPUs in both systems are not utilised by F@H cores.
I've have now transferred the primary card [A] to the GPU case, connected via a PCIE3-x1 riser, but still configured as the primary [display] card.
I'm using Windows 10Pro. Motherboard is an ASRock B450 Pro4.

[X-Wing]

Loving Core 22. Went from seeing max of just over 800k ppd to just now seeing around 840k.

[foldy]

@BlazingDragon: That's it Radeon VII loves these projects 11738/11739 because of high atom count it gives high PPD on these high shader count GPUs.

[BlazingDragon]

@BlazingDragon: That's it Radeon VII loves these projects 11738/11739 because of high atom count it gives high PPD on these high shader count GPUs.

Agreed.

The Radeon VIIs really seem to like the 11738 and 11739 WUs, but it should be noted that (based on my results) the 11737 has disappointing PPD (~780,000) on the Radeon VII.

Also interesting that there didn't seem to be much of a PCIE riser penalty on the 11737 WUs [even under Windows].

[foldy]

Project 11737 has too few atom count to fully utilize all shaders of Radeon VII. Projects 11738 and 11739 have big atom count that is the difference.

[snapshot]

On my temperature-limited GTX 1660 Ti, 11738 is giving 800k PPD where 1426x WUs only give 560k.
I'd give more details of the other Core 22 WUs but HFM's benchmark viewer isn't working and I haven't got round to re-installing it.

[bruce]

To successfully benchmark small atom-count projects on a wide variety of GPUs (small and large), thee project owner needs to tweak his project settings.

Small atom-count projects depend almost entirely on the GPUs shader-count and clock rate. Large atom-count projects generally need wider bandwidth so they tend to have degraded performance on Windows and/or with 1x risers.