The UofAi Benchmark Standard (UBS-1)

The operational Super-AI definition

There is no scientific consensus definition of superintelligence, so the SHI publishes an operational proxy and measures against it. The index reads 100 when a single system simultaneously: (a) matches human domain-expert accuracy (~90%) on the hardest closed-ended academic benchmark in active use (currently Humanity's Last Exam); (b) matches human performance on novel-skill-acquisition benchmarks designed to be easy for humans and hard for AI (currently ARC-AGI-2 and ARC-AGI-3, human baseline 100%); (c) reaches a 50%-reliability autonomous task horizon of one work-month (~167 hours) on METR's methodology; and (d) performs frontier-researcher-level AI R&D on public evals.

Superintelligence Horizon Index (SHI)

The Superintelligence Horizon Index is a directional composite built from public benchmarks against an operational definition — it is not a scientific measurement of distance to superintelligence, for which no consensus definition exists. V1 averages three equally-weighted subscales; a research-automation subscale enters when a verifiable public datum is curated, at which point the weights rebalance and the change is logged.

shiExpertKnowledge  = hleSotaPct / 90 * 100                 // clamped [0,100]
shiNovelReasoning   = mean(arc2Pct, arc3Pct)               // human baseline 100
shiAutonomyHorizon  = 100 * ln(horizonMinutes) / ln(10020) // 167h -> 100
SHI                 = mean(subscales)                       // clamped [0,100]

UofAi Compute Pulse (UCP)

A log-anchored reading of frontier training compute. The GPT-4-era run (2×10²⁵ FLOP, 2023) anchors to 100; every +100 points is one additional order of magnitude. Four sub-gauges (compute stock, hardware efficiency, power) are displayed but not folded into the headline.

UCP = 100 + 100 * log10(frontierFlop / 2e25)

Human Parity Tracker (HPT)

A ledger of falsifiable milestones across ten domains, each marked Achieved, Frontier, or Open. Frontier milestones count half.

HPT = (achieved + 0.5 * frontier) / total * 100

Source hierarchy

Primary measurement organizations — METR, Epoch AI, ARC Prize Foundation, the CAIS/Scale HLE leaderboard, Stanford HAI AI Index, Artificial Analysis, MLCommons/MLPerf, TOP500, IEA — outrank lab self-reports, which outrank press coverage. Lab self-reports are always flagged as such.

Data classes

Every datum is tagged: measured (directly observed on a benchmark), extrapolated (projected from a measured trend), or estimated (a best available approximation where no direct measurement exists). Forecasts are always labelled extrapolations; the SHI is always shown with its band, never as false precision.

Conflict rule

When credible sources disagree — as they currently do on HLE state of the art — the Pulse publishes the range and both citations rather than silently picking one.

Refresh cadence

Indices refresh quarterly with a public changelog; the Signal Feed refreshes monthly. Every surface shows an “as of” date as a stale-data guard. Historical snapshots are never edited; corrections ship as new snapshots noted in the changelog.

Saturation-rotation rule

When a component benchmark saturates (state of the art above 90% of the human baseline for two consecutive quarterly refreshes), it is frozen into the Human Parity Tracker as an achieved milestone and replaced by its published successor (for example HLE by its successor; ARC-AGI-2 → ARC-AGI-3 → ARC-AGI-4). Every substitution is logged with a re-basing note, so the SHI stays a living instrument rather than dying of saturation the way MMLU and GPQA did.

Latest snapshot changelog

• UCP (as of 2026-07-02)

  v1 initial reading. Anchor: GPT-4-era frontier run (2e25 FLOP, 2023) = 100.

• SHI (as of 2026-07-02)

  v1 initial reading. Research-automation subscale (AI R&D evals) pending a verifiable public datum; composite currently averages three subscales and will rebalance when added (change will be logged).

• HPT (as of 2026-07-02)

  v1 ledger, 24 milestones across 10 domains. Weights equal in v1.