Benchmark models independently on identical hardware and publish a simple comparable efficiency label

The proposal

Run every model through the same task suite on fixed, standardised hardware, isolating GPU energy, then bucket the results into a small number of star bands and publish them as a leaderboard plus a shareable label. Test open models automatically, and offer closed models a secured sandbox so proprietary weights never leave a controlled environment.

Why it works

A standardised, independent benchmark removes the confound of different hardware and self-reported numbers, producing genuinely comparable ratings. A simple label makes efficiency legible to non-experts and usable in procurement, creating market and reputational pressure on providers to disclose.

Where it fits

Model selection and procurement, model routing (send easy queries to efficient models), and policy or corporate reporting that wants a defensible efficiency signal.

Trade-offs and limitations

Without closed-model participation the ranges stay incomplete, so the mechanism ultimately depends on voluntary disclosure or regulation to reach full coverage. Fixed-hardware benchmarks measure relative efficiency under controlled conditions, not the exact energy of any given production deployment.