FlashProve: What Anthropic's Provable Inference Speed Measures

Anthropic says a single H100 can generate proofs for a 400B-parameter model at about 110 tokens per second. That number comes from the company's Nov. 2 research post on FlashProve, an open-source prototype for provable inference, and it's the figure most coverage repeats. The repository behind it says what the 110 is counting, and it isn't a model answering a user.

Provable inference means a provider commits to one set of model weights, then attaches a cryptographic proof to each reply showing those weights produced it. The user checks the proof on an ordinary CPU, without seeing the weights. It targets a real gap, which Anthropic describes in the post: a user "cannot check that the responses came from the specific model that a developer claims is responding, as opposed to one secretly substituted in by a routing service for cost reasons."

Anthropic's announcement on X, Nov. 3, 2026, introducing FlashProve (@AnthropicAI)

View on X

The X post opens: "New Anthropic research: progress on provable inference for large language models." It says FlashProve "allows developers to verify whether a given response came from a given AI model."

The 110 is a prompt-reading speed

The write-up lists the setup: Llama 4 Maverick, a 400B mixture-of-experts model that uses about 17B parameters per token, in a third party's 4-bit conversion; one H100; proofs of about 19 MB for each 1,024-token request. A batch of 24 requests took the prover about 223 seconds. Do the division: 24 × 1,024 = 24,576 tokens, over 223 seconds, is 110 tokens a second. So the rate counts every token of every prompt.

The repository's performance guide is plainer than the post. Its figures are "steady prefill rates: each request's 1,024 prompt tokens count, nothing is generated." The post adds that "each proof checks one token." And the speed-up claim, 70 to 95 times the GPU time of vLLM, compares proving against vLLM's prefill, the step where a model reads a prompt. The guide notes that its 20.1× comparison against decoding "is not the cost of proving a generated reply."

Our reading: 110 tokens per second is a throughput for proving a pass over text you already have. It isn't the speed at which a chat or an agent would get proved answers, and Anthropic never claims it is.

The part where real replies fail

The weights Anthropic proves are an integer version of the model. In the guide's words, its top token "differs from the 16-bit model's at 6.81% of positions," measured with a Python implementation rather than the released programs, with no task benchmark behind it.

That matters because of the next test. In the guide's "Proving, not serving" paragraph, stored replies from stock vLLM, 96 at 16-bit and 96 at FP8, were scored against the integer model at exact match. "None did" pass. The team got 48 of 48 short replies through when vLLM ran with a plug-in built to compute the integer model exactly, at 0.58 to 1.46 seconds per token, and they mark that result as not produced by the released programs.

Anthropic's post states the same limit: "A reply from a floating-point implementation, or sampled decoding, could depart at some position and no longer exactly match the computation proved."

So the proof covers greedy decoding on the integer model. A normal production reply, sampled from a floating-point stack, is not what it certifies today. The 6.81% figure is a measurement of the model, not of how many real replies would fail.

What a yes tells you

The repository's Known limits page contains the most useful sentences in the whole release.

• The provider picks the model. A yes "does not say whose weights they are." A provider can commit to a small or degenerate model, and the page advises taking the commitment ID "from a party you trust... never from the server you check."
• The looser serving policy proves less. Serving uses a "clipped gap" test, and cp_verify then prints WITHIN LIMITS instead of YES. On loose limits, the page warns that "almost any output passes" and that this isn't evidence the reply is the model's.
• Hiding is expected, not shown. The authors "expect, rather than know, that the programs hide the weights." They add that "few attacks were sought."

Soundness, the property that matters most for a verifier, is the strongest part: proved in Lean for the Lean verifier, with the C++ verifier tested against it. The post also says the fast C++ code "carries no formal guarantee."

The bill for checking

The post gives the average proof as 18.78 MB. Checking one takes 17 seconds on 24 threads and 21 seconds on one. The prover spends about 9.2 seconds per request at the 24-request batch, according to the guide, so we calculate that checking one proof on a single thread takes roughly twice as long as producing it. The vLLM middleware check for a batch of 24 took 94 seconds in the authors' tests.

The hardware list is long. The guide records about 270 GB of host memory while serving and about 300 GB while committing, next to an 80 GB GPU. Committing the weights takes about 39 minutes, and a commitment is good for roughly 16 million requests before the provider has to commit again.

Who has the receipts

For outside context, the post's prior-work section is the place to look: VerInf, the only earlier proof of the same model Anthropic knows of, reports 14.3 hours for 1,000 tokens on a DGX Spark with a 93.6 GB proof. Attestable reports 53 to 77 tokens per second for Gemma 4 31B, a dense model, and, per Anthropic, has not released a matching open-source library. Different models, different hardware, different rules: not a ranking.

Two things bear on trust. The guide says the timed runs' records and the measuring scripts "are not in this repository," so nobody can rerun the headline numbers. And the README describes the project as "A research artifact: not maintained, not accepting contributions."

The post is also open about who wrote what. It credits Claude "has produced improvements on zkSNARKs for inference," and that the protocol specification is "Claude-written." Anthropic's response to the risk of an AI making hard-to-audit decisions is to publish Lean statements that people have to read by hand.

Why Anthropic published it

The post ties provable inference to Anthropic's Frontier Safety Roadmap, which set a goal of a prototype by Sept. 30, 2026. By Anthropic's account, it met that goal with a separate project that signs its inference traffic and recomputes inference on a trusted server. On Oct. 19 it renamed the goal "inference verification," because "provable inference" suggested "a narrower approach than we believe is necessary." FlashProve is one route toward that goal, and the post calls it intermediate progress and "not a completed research agenda."

It also points to regulation: a regulator could ask a provider for proofs that approved models were served, though the post notes this "does not establish that the hardware is not, e.g., training on the side."

Questions before you test it

Can I check my router's model with it today? Not in a meaningful way. Contexts top out at 1,024 tokens, and sampled or floating-point replies aren't covered.

Does the proof hide the architecture? No. According to the post, the architecture isn't hidden, and hiding it is future work.

Will Anthropic use it in Claude's API? Unconfirmed. The post doesn't announce an integration.

Is the speed independently measured? Unconfirmed for FlashProve. Attestable's figures are independent of Anthropic, but they are for a different model.

A line to check first

If a vendor ever shows you a green check for "verified model," ask where the commitment ID came from and which acceptance policy ran. For FlashProve, those are the two lines that say whether YES means exactly what it sounds like.

Source: Anthropic, Progress on provable inference for large language models

anthropic flashprove provable inference zero knowledge llama 4 maverick

AI

About the author

AI Briefing Daily Editorial. The editorial desk of AI Briefing Daily, an independent publication covering artificial intelligence. Every story is researched against primary sources, checked by hand and reviewed by a person before publication.