FAHBench feature proposals & brainstorming

[NoMoreQuarantine]

I know that FAHBench isn't priority right now, but in my singular mission to provide the people with accurate performance data, I have a couple of proposals and thoughts for features to discuss.

• Add Gromacs core
• Create default test profiles for GPUs & CPUs. Make it simple and consistent by default with advanced options available to those who know what they're doing
• Integrate with database for reference & comparison (I plan to do a trade study on this in the coming days)
• Record all relevant system information:
  • Motherboard make & model; chipset & rev; BIOS version; CPU model & frequency; RAM type, quantity, & frequency
  • Video card make & model; BIOS version; bus interface; driver version; GPU model & frequency; VRAM type, quantity, & frequency
  • Operating system type & version
  • FAHBench version, OpenMM or Gromacs version
  • Processor temperatures before, during, and after testing. The ideal would be to detect thermal throttling of the processor. Maybe even get data on performance variance over temperature ranges; I can dream at least
• Integrate FAHBench into advanced control. Medium term it could just be a button that launches FAHBench. Long term, it may be possible to integrate with FAHClient for system optimization and providing information needed for the "streaming" that has been discussed elsewhere, as well as load balancing workload assignments to the clients in general

Questions & thoughts:
Is there a convenient way to tell that hardware is overclocked without needing to maintain a separate database of stock specs? How do we deal with future core changes? Can we make it "futureproof" by measuring general performance across the board and then estimating how it would perform in OpenMM & Gromacs? If not, perhaps there is a way to estimate how processors only tested with the previous versions may perform using "representative" processors that have been tested with both versions. What other issues, considerations, and features have I missed?

[PantherX]

Those are some pretty cool ideas. When it comes to system component detection, it won't be wise for the F@H Team to re-invent the wheel. That's a massively complex undertaking. However, it would be a lot easier if they can use an existing system like via a partnership with HWiNFO (https://www.hwinfo.com/about-software/) to get most of those data points.

Maintaining a database of hundreds (initially) and potentially several thousands (future) of system would require a massive amount of investment and also maintenance. I wonder if F@H can have a partnership with 3DMark (https://www.3dmark.com/) and have a module for "Distributed Computing" where F@H can be part of the testing suite and uses the existing database without the hassle of creating and maintaining one.

I currently like how modular the V7 client is. Thus, rather than fully integrating FAHBench into FAHControl, I would like them to be packaged together for ease but it should be independent of each other. By making these modules separate, maintaining and contributing would be easier as opposed to a single application with everything built into it.

[NoMoreQuarantine]

Modularity and tool reuse is absolutely the way to go. Partnerships would be great and there are probably some open source projects we could pull from as well.

[foldinghomealone2]

I don't understand why my mainboard information would be of any relevance to FahBench.
And it's not about creatng databases it's about folding.
FahBench gives quite interesting numbers but they are an indication only. Like what GPU is faster under average conditions.
This mainboard (except for pcie connection) doesn't matter.
What matters however to know is whether the GPU was heated through before the test. Results from 15min tests are roughly 5% lower than for 1min tests (but that drop could be higher in systems with bad cooling).

But still what you get is a seemingly very accurate number.

However, real life folding has it's own rules and your PPD will vary about 20% from WU to WU with the very same GPU and folding conditions.
Don't make everything too complicated.

[NoMoreQuarantine]

I don't understand why my mainboard information would be of any relevance to FahBench

It's system variables that may affect performance. You are right though, make and model probably isn't that useful. Chipset & BIOS are also arguable.

it's not about creating databases it's about folding.

That's what it all comes down to in the end.

Results from 15min tests are roughly 5% lower than for 1min tests (but that drop could be higher in systems with bad cooling).

It would make sense to run the test for long enough to either reach a steady temp or start throttling. Tell the user if it begins throttling and for people who want to submit to the database, throttled results should be ignored.

[foldy]

CPU infos are interesting, e.g. below 2 Ghz it cannot fully feed a GPU.
Each GPU FAH core needs 1.5 GB RAM on Windows so for many GPU rigs RAM size may bottleneck so interesting too.
PCIE slot bandwidth is interesting cause it can bottleneck too.

So if I have a system with 8GB RAM, 4-core CPU 2GHz and 4 GPUs at pcie 3.0 x4 running Linux then this is optimal.

[NuovaApe]

Those are some pretty cool ideas. When it comes to system component detection, it won't be wise for the F@H Team to re-invent the wheel. That's a massively complex undertaking. However, it would be a lot easier if they can use an existing system like via a partnership with HWiNFO (https://www.hwinfo.com/about-software/) to get most of those data points.

Agreed - cool ideas. I'd love a repeatable test environment; CPU/GPU/both. Multiple GPUs for the hardcore out there.

GPU-Z (TechPowerUp) and CPU-Z (CPUID) both offer SDKs/dlls for obtaining those hardware details, but guessing for Windows only.
I'd be wary of adding similar into FAHBench; these singleton bits of hardware typically only allow singleton monitoring.
If FAHBench "takes that connection" then overclockers won't be able to run their favorite hardware monitoring tools.

I wonder if F@H can have a partnership with 3DMark (https://www.3dmark.com/) and have a module for "Distributed Computing" where F@H can be part of the testing suite and uses the existing database without the hassle of creating and maintaining one.

That's a damn fine idea! Those 3D wizards might even have resources to assign to FAHViewer. They'd want to show something snazzy and graphical anyways in their OpenCL benchmark (must be optional though). Plus 3DMark has built in hardware monitoring - but you have to pay $29.99 for the version with HW monitoring, PCI features test, stress test etc.

If 3DMark could add FAHBench to the free version they would have something of true merit in their portfolio - science. Surely a win-win?

[foldinghomealone2]

I don't see why such an effort should be necessary.

I see only two puroposes of a FahBench database
1. A ranking GPUs used for folding to choose the next GPU.
2. To compare my GPU to other (same or similar) GPUs to see if my GPU performs as usual.

But it's not interesting to know if a GPU scores 80.012 on one board and 80.135 on the other board.
However when my GPU scores only 72.035 then I would like to know why. And then I would go to this forum and ask.