HW-Bench: What 15 Models' Antenna-Design Scores Show

Arena Physica released HW-Bench on Nov. 4, 2026, a benchmark that scores frontier models on hardware engineering work. Its first and only job is designing a 2.45 GHz antenna, and on that job GPT-6 Astra passes 67% of runs while Claude Opus 5 passes 33% and Kimi K3 passes 7%.

Arena Physica's post announcing HW-Bench, Nov. 4, 2026 (@arenaphysica)

View on X

The post calls HW-Bench "the first hardware engineering benchmark built in partnership with the world's most advanced hardware companies." The leaderboard lists 15 models and carries an Oct. 29 update date, six days before the post.

What the scores count

The page's own caveat comes first: "Early results based on six tasks within one job... Rankings may shift as we add jobs." Each task runs five times per model. The headline metric, pass^k, is "the chance that k random runs (of 5) of a task all pass every critical criterion." Accuracy is a separate, weighted share of criteria passed from 0 to 1.

Six tasks times five runs is 30 runs per model, by our arithmetic, so one run moves a score by about 3.3 points. GPT-6 Astra's 67% works out to 20 of 30.

Accuracy that doesn't pass

The page warns that "high accuracy score can still mean low pass^k, since failing one critical criterion fails the run." The table shows it plainly:

• GPT-6 Astra: accuracy 0.81, and 67% at pass^1, pass^3 and pass^5.
• GPT-6.1 Sol: accuracy 0.78, falling from 57% to 42% to 33%.
• Claude Opus 5: accuracy 0.71, falling from 33% to 3% to 0%.
• GPT-6 Sol: accuracy 0.71, falling from 37% to 13% to 0%.

Same 0.71 accuracy, different reliability. Only Astra held its score when every run had to pass; 4 of the 15 models reach more than 0% at pass^5, and Astra is the only one above 33%.

Cost and time per job

Astra's mean cost is $50.49 per job (95% interval $45.45 to $55.60) and 1 hour 28 minutes. GPT-6 Luna costs $0.55, about 92 times less, and passes 30% of single runs at accuracy 0.59. GPT-6.1 Sol sits between at $8.42. One quirk: the table column says cost per job, while the metric definition says "mean USD per run of a task," so check the unit before comparing with your own budget.

Kimi K3 ranks 14th with 7% at pass^1, accuracy 0.40 and the longest time, 3 hours 10 minutes. It's the earlier version of the model whose coding benchmark scores and license we read this week, and the page doesn't list K3.1 Pro.

Who built the exam

The about page says Arena Physica also makes Atlas, a harness for hardware engineering, and Heaviside, an electromagnetics foundation model. Each task has a prompt, input files in native formats, engineer tools, an expert golden answer and a rubric. Verifiable outputs are checked in code, and subjective criteria go to an LLM judge "calibrated by human engineers."

The founding partners that review every task and rubric are unnamed ("coming soon"), and the partners' production data "stays private to them." The page states no license and no route to download the tasks. That doesn't make the numbers wrong, but nobody outside can rerun them yet. When an examiner also writes the questions, the useful question is what a third party can reproduce.

Before you use it to pick a model

Treat the table as one antenna job, not as hardware engineering in general. Check pass^5 first if your workflow can't accept a failed run, then look at cost per job. And watch the jobs section for "Build & test" and "Operate & sustain," which the page lists with no results yet.

Source: Arena Physica, HW-Bench leaderboard

hw bench arena physica hardware engineering ai benchmarks gpt 6 astra

AI

About the author

AI Briefing Daily Editorial. The editorial desk of AI Briefing Daily, an independent publication covering artificial intelligence. Every story is researched against primary sources, checked by hand and reviewed by a person before publication.