Audited July 9, 2026

Frontier Model Benchmark Matrix

Public numeric capability results for GPT-5.6 Sol, Claude Mythos and Fable 5, Muse Spark 1.1, and Grok 4.5. NR means no public numeric result was found in the audited sources.

A benchmark is only one projection of model behavior. Increasingly, a compelling real-world demo, such as watching a model build a game, can feel more persuasive than another leaderboard point because people care whether a model is capable, reliable, and pleasant to work with. That instinct is a useful response to Goodhart's law, but science needs more than vibes: the challenge is to turn those real-use qualities into repeatable evaluations.

TL;DR

Reporting choices shape the story: For 68 capability benchmark rows in the current matrix, 32 are unique; 36 are from two models +; only 3 appear in all four. Safety reporting is even more fragmented: 71 of 80 normalized rows appear once in the current four releases.

Evaluation is moving toward real work. New additions concentrate in coding, professional tasks, agents/tools, and science rather than another round of static exams.

Benchmark reporting turns over quickly. Across 14 same-lab release transitions, only 56% of previously reported benchmark editions appear again in the next report; 44% are not carried into the next public report, though that does not prove the evaluation itself was retired.

Current releases

Public numeric results from the four current release bundles. NR means no public numeric result was found in the audited source.

Reporting coverage is not model quality. More reported benchmark rows do not imply a better model, and NR does not establish that an evaluation was never run.

Benchmark rows: 68
Unique to one release: 32
Shared across 2+ releases: 36
All four reports: 3

Benchmark	GPT-5.6Sol	Claude 5Mythos / Fable	Muse Spark1.1	Grok4.5
Professional
Agents' Last Exam	52.7	40.5 F	NR	NR
GDPval-AA v2*	1747.8	1759.6 F	1381	NR
AA Intelligence Index v4.1	58.9	59.9 F	NR	NR
Management Consulting (internal)	43.2	35.5 F	NR	NR
Finance Agent v2	NR	56.3 F	57.2	NR
Big Finance Bench	53.0	NR	NR	NR
JobBench	NR	NR	54.7	NR
OfficeQA Pro	NR	57.9 F	NR	NR
Legal Agent (Harvey held-out)	NR	13.3 F	NR	NR
Legal Agent (full public)	NR	16.9 M	NR	NR
Agents + tools
BrowseComp	90.4	88.0 M	NR	NR
BrowseComp multi-agent*	92.2 U	93.3 M	NR	NR
MCP Atlas	NR	83.3 F	88.1	NR
Toolathlon	58.0	61.7 M	NR	NR
Toolathlon-Verified	NR	NR	75.6	NR
AutomationBench	18.1	17.4 F	NR	NR
OSWorld 2.0*	62.6	NR	14.2 / 47.3	NR
OSWorld-Verified	NR	85.0 M/F	80.8	NR
WebArena-Verified	NR	NR	69.0	NR
DeepSearchQA*	NR	94.2 M	84.9	NR
Coding
SWE-Bench Pro	64.6	80.3 M	61.5	64.7
Terminal-Bench 2.1	88.8	88.0 M	80.0	83.3
DeepSWE 1.1	72.7	69.7 F	53.3	53.0
DeepSWE 1.0	NR	66.1 F	NR	62.0
SWE Marathon	NR	24.0 F	NR	29.0
AA Coding Agent Index v1.1	80.0	77.2 F	NR	NR
SWE-Bench Verified	NR	95.5 M	NR	NR
FrontierCode Diamond	NR	29.3 F	NR	NR
CritPt	NR	28.6 M	NR	NR
ArxivMath	NR	78.5 M	NR	NR
RiemannBench	NR	55.0 M	NR	NR
AI self-improvement
Internal Research Debugging	68.3	NR	NR	NR
KernelGen 1P	61.1	NR	NR	NR
NanoGPT	9.69	NR	NR	NR
PostTrainBench Lite	50.3	NR	NR	NR
RSI Index	57.9	NR	NR	NR

Benchmark	GPT-5.6Sol	Claude 5Mythos / Fable	Muse Spark1.1	Grok4.5
Reasoning + long context
Humanity's Last Exam (no tools)*	NR	59.0 M	52.2	NR
Humanity's Last Exam (with tools)*	NR	64.5 M	62.1	NR
GPQA Diamond	94.6	94.1 M	NR	NR
FrontierMath Tier 1-3 v2	89.0	87.0 F	NR	NR
FrontierMath Tier 4 v2	83.0	87.8 F	NR	NR
MRCR 256K-512K	91.5	NR	NR	NR
MRCR 512K-1M*	73.8	NR	54.1	NR
GraphWalks BFS 256K	90.7	91.1 M	NR	NR
GraphWalks BFS 1M	77.1	79.4 M	NR	NR
GraphWalks Parents 256K	NR	99.96 M	NR	NR
ARC-AGI-3	7.78	NR	NR	NR
Science + health
HealthBench Professional*	60.5	66.0 M	59.3	NR
HealthBench*	57.0	62.7 M	NR	NR
GeneBench Pro	28.7	NR	NR	NR
LifeSciBench	59.9	NR	NR	NR
MedChemBench (internal)	48.3	NR	NR	NR
BioMysteryBench (hard)	NR	46.1 M	NR	NR
BioMysteryBench (human solved)	NR	83.9 M	NR	NR
Multimodal
gdp.pdf	30.7	29.8 F	NR	NR
BenchCAD	70.6	38.4 M	NR	NR
BenchCAD + Python	83.4	65.0 M	NR	NR
CharXiv Reasoning (no tools)	NR	88.9 M	NR	NR
CharXiv Reasoning (with tools)	NR	93.5 M	88.4	NR
BabyVision (with tools)	NR	NR	76.3	NR
Blueprint-Bench 2	NR	38.6 M/F	NR	NR
MMMU Pro (no tools)	83.0	NR	NR	NR
MMMU Pro (with tools)	84.6	NR	NR	NR
Cybersecurity
CyberGym	84.5	83.8 M	59.0	NR
ExploitBench	73.5	78.0 M	NR	NR
ExploitGym (2h)	24.9	NR	0.6	NR
Capture-the-Flag Challenges	96.7	NR	NR	NR
SEC-Bench Pro	71.2	NR	NR	NR

Safety reporting

Evaluation rows: 80
Safety dimensions: 6
Release safety reports: 3 / 4

Safety takeaway

Safety reporting is broad but highly fragmented. Of 80 normalized evaluation rows, 71 appear in only one of the three current release documents with quantified safety results. Only VCT and HealthBench Professional appear in all three. These counts measure public disclosure overlap, not which model is safest or how much safety testing each lab performed.

VCT / Multimodal Troubleshooting Virology

Public benchmark

Expert-level multimodal questions about virology knowledge and protocol troubleshooting.

GPT-5.6: Sec. 9.1.1.1, Sec. 9.1.1.8 Claude 5: Sec. 2.2.4.1 Muse Spark: Table 1

Safety evaluation	GPT-5.6Sol27 reported	Claude 5Mythos / Fable31 reported	Muse Spark1.133 reported	Grok4.50 reported
Dangerous capability
				-
		-	-	-
	-	-		-
		-	-	-
		-	-	-
	-		-	-
	-		-	-
			-	-
		-		-
		-		-
		-	-	-
	-	-		-
	-	-		-
	-	-		-
	-	-		-
	-	-		-
	-	-		-
Harm refusal
		-	-	-
	-		-	-
	-		-	-
	-		-	-
	-	-		-
	-	-		-
	-	-		-
	-	-		-
	-	-		-
	-	-		-
	-	-		-
Adversarial robustness
		-	-	-
		-	-	-
		-	-	-
		-	-	-
	-	-		-
	-	-		-
	-	-		-
	-	-		-
	-			-
	-		-	-
Control + authorization
		-	-	-
		-	-	-
	-		-	-
	-		-	-
	-		-	-
	-		-	-
	-	-		-
Alignment + oversight
		-	-	-
		-	-	-
		-	-	-
		-	-	-
		-	-	-
		-	-	-
	-		-	-
	-		-	-
	-		-	-
	-		-	-
	-			-
	-		-	-
	-		-	-
	-		-	-
	-			-
	-		-	-
	-	-		-
	-	-		-
	-	-		-
	-	-		-
	-	-		-
	-	-		-
	-	-		-
Human impact
		-	-	-
		-	-	-
		-	-	-
				-
			-	-
	-		-	-
	-		-	-
	-		-	-
	-		-	-
	-		-	-
	-		-	-
		-	-	-

Where the categories come from

These six categories are an editorial normalization for this audit, not a shared taxonomy endorsed by the labs. They align OpenAI's Model Safety, Alignment, and Preparedness sections; Anthropic's Safeguards, Agentic Safety, Alignment, and Responsible Scaling Policy sections; Meta's Advanced AI Scaling Framework, Adversarial Robustness, and Model Behavior scorecards; and the malicious-use, loss-of-control, and dual-use framing in xAI's earlier Grok 4.20 system card.

Control + authorization asks whether a tool-using model stays within the user's intent, seeks consent for consequential actions, and avoids destructive or malicious acts. Human impact groups mental health, child safety, health, bias, election integrity, and related effects on people and groups.

A check means the audited release document published a numeric result or quantified rate. Metrics and protocols are usually not directly comparable, so this map intentionally does not crown a safety winner. Cyber and AI self-improvement capability benchmarks already listed in the current matrix are not duplicated here.

No quantified safety result was found in the audited Grok 4.5 launch post; that does not show no safety testing occurred. xAI's earlier Grok 4.20 system card did report safety evaluations; none are attributed to 4.5 here.

Reporting history

Benchmark reporting history, July 2025 to July 2026

An edition-level audit of 18 first-party flagship release bundles. Each cell answers one question: did that release publicly report a numeric result for this benchmark edition? Run details remain attached to every reported cell.

Release bundles: 18
Benchmark editions: 143
One-report editions: 70

Release-to-release continuity

Retention is the share of the previous report's benchmark editions that appears again in the next report.

+ new - dropped

OpenAI

6 reports

23 editions

baseline

5.1

+1 -22

5.2

50%

+23 -1

5.4

63%

+8 -9

5.5

78%

+6 -5

5.6

38%

+26 -15

Claude

6 reports

4.5

21 editions

baseline

4.6

71%

+15 -6

Mythos P

37%

+5 -19

4.7

94%

+18 -1

4.8

79%

+15 -7

Claude 5

93%

+13 -3

Muse

2 reports

1.0

20 editions

baseline

1.1

25%

+14 -15

Grok

4 reports

8 editions

baseline

4.1

+5 -8

4.3

+4 -5

4.5

+6 -4

Benchmark reporting timeline

Lab

CategoryReportingSearch

143 rows

Selected benchmark

Humanity's Last Exam

Reasoning · 13 of 18 scoped reports

OpenAI

5 / 5.2 / 5.4 / 5.5

Absent from latest

Claude

4.5 / 4.6 / Mythos P / 4.7 / 4.8 / Claude 5

Present in latest

Muse

1.0 / 1.1

Present in latest

Grok

Absent from latest

	OpenAI						Anthropic						Meta		SpaceXAI
Benchmark / edition	5Aug '25	5.1Nov '25	5.2Dec '25	5.4Mar '26	5.5Apr '26	5.6Jul '26	4.5Nov '25	4.6Feb '26	Mythos PApr '26	4.7Apr '26	4.8May '26	Claude 5Jun '26	1.0Apr '26	1.1Jul '26	4Jul '25	4.1Nov '25	4.3Jun '26	4.5Jul '26

reported—not reported

What the timeline captures

Distinct named editions are separate rows, such as SWE-Bench Verified versus Pro, OSWorld-Verified versus OSWorld 2.0, and Terminal-Bench 2.0 versus 2.1. Run-setting differences such as tool access, reasoning effort, context length, or agent scaffolding stay in the cell detail. A missing cell means no public numeric result was found in that release bundle, not that the lab stopped evaluating the benchmark internally.

Audit boundary

The corpus covers flagship general-purpose releases and their principal first-party capability tables or chapters. Smaller, fast, and domain-specialized model launches are excluded, as are refusal, alignment, personality, and preparedness-only evaluations. This keeps the retention denominator comparable while still including named cyber and life-science capability benchmarks reported in capability sections.

Source corpus (18 release bundles)

OpenAI

GPT-5 Aug 2025
GPT-5.1 Nov 2025
GPT-5.2 Dec 2025
GPT-5.4 Mar 2026
GPT-5.5 Apr 2026
GPT-5.6 Sol Jul 2026

Anthropic