Key information
Modders have successfully bypassed firmware restrictions on NVIDIA GeForce RTX 30-series and 40-series graphics cards to unlock compute potential previously reserved for professional-grade hardware. By overriding internal limitations, these modifications allow consumer GPUs to achieve instruction issuing rates comparable to their professional counterparts.
For the Ampere-based RTX 30-series, researchers developed a method to double the instruction issuing speed of FP/BF16 FMA into FP32 operations. This modification enables cards like the RTX 3090 Ti to perform matrix multiplication up to 80% faster, significantly benefiting neural network applications.
The Ada Lovelace-based RTX 40-series saw a 2x speedup in FP16 and BF16 tensor throughput. Testing on an RTX 4090 demonstrated an increase from 165.6 TeraFLOPS to 331.6 TeraFLOPS, matching performance levels typically restricted to the professional RTX 6000 Ada Lovelace series.
These modifications maintain compatibility with stock NVIDIA drivers and operate within original power limits, such as the 450 W TGP. While the unlocks improve token throughput for local large language models, users must enable the mods at boot. Importantly, these modifications void the GPU warranty.
