Xiaomi-OCR-0: Why a 95.24 Real5 Score Became 92.73

Xiaomi-OCR-0 has two scores on the same benchmark, and a public thread on Hugging Face shows where the gap comes from. Xiaomi's technical report gives the 0.8B document-parsing model 95.24 on Real5-OmniDocBench, "the highest among the compared methods." The benchmark's own leaderboard lists 92.73 and ranks it fourth. Both numbers come from the same two-stage outputs. The difference, according to the benchmark's maintainers, is the version of the scoring code.

Xiaomi Tech's post on X, Oct. 30, 2026, announcing the open-source Xiaomi-OCR-0 document model (@XiaomiTech_)

View on X

The post from @XiaomiTech_, timestamped 1:05 PM on Oct. 30, introduces "an open-source document understanding model with just 0.8B parameters" that scores "95.24 on Real5-OmniDocBench and 83.2 on average across five OCR-oriented VQA benchmarks." It adds that the model "ranks first in document parsing performance among models with fewer than 1B parameters" on OmniDocBench v1.6, and that weights, inference code, a demo, an MCP service and an Agent Skill are open source.

It wasn't a first release. The model card on Hugging Face says the technical report and weights came out on Sept. 30, a month before the post, and the paper was submitted to arXiv on Sept. 28.

Two numbers, one benchmark

The gap is 2.51 points. Xiaomi's report says its Real5 numbers come from two-stage inference: "PP-DocLayoutV3 detects regions, Xiaomi-OCR-0 recognizes each crop, and the predictions are assembled into a page-level output." The reported 0.8B parameter count "excludes the external layout detector." So 95.24 describes a system of two models.

That isn't what separates the two scores. In a pull request on Xiaomi's own repository, a researcher from the team behind the Real5 benchmark wrote that the benchmark is aligned with OmniDocBench v1.5, that results "produced with" the newer v1.6 code "are not directly comparable with those of other models on the existing leaderboard," and that the version difference is the primary reason for discrepancies. When Xiaomi asked whether the leaderboard run used whole-page inference, the researcher replied that it was "exactly Two-stage with PP-DocLayoutV3," using the repository's own code.

Xiaomi then shared its original two-stage predictions. On Oct. 16 the researcher scored them with the v1.5 code and got 92.7863 overall, which they called "broadly consistent" with the 92.73 submitted. Xiaomi merged the pull request on Oct. 23. Xiaomi's evaluation guide confirms its own side: "Both approaches use the OmniDocBench v1.6 evaluation code."

Which number counts

Our reading: both numbers are real, and they answer different questions. 95.24 is Xiaomi's score under the v1.6 scorer. 92.73 is the score under the v1.5 scorer that the Real5 leaderboard and its other rows use. The paper's Real5 table puts 95.24 next to rivals' figures that match the leaderboard (PaddleOCR-VL-1.6 at 93.19, OvisOCR2 at 92.29, GLM-OCR at 90.32), and the benchmark team says v1.6 results aren't comparable with those. So "highest among the compared methods" holds on Xiaomi's table, and the leaderboard's like-for-like reading puts it fourth. As of Nov. 3 the model card and the Oct. 30 post still show 95.24 and don't mention the scorer version.

Two caveats. PaddlePaddle hosts that leaderboard and makes a competitor, PaddleOCR-VL, though the researchers published their evaluation files. And the standings may move: on Oct. 23 another member of that team wrote that Real5 will be upgraded to match the newest OmniDocBench, with new metrics to follow. MiniCPM-OCR-1.0 (93.75) and OvisOCR3 (93.71) sit above Xiaomi today; both were added Nov. 2.

Where it leads and where it doesn't

The model card is more modest than the post. It labels the Real5 score "#1" and calls the OmniDocBench v1.6 (96.83) and Wild (87.94) results "#2." On v1.6 it sits 0.04 points behind TeleOCR's 96.87, a 1.2B model, and on Wild it trails TeleOCR's 88.53. "First among models under 1B" is true on those two tables; "first" isn't.

On the five OCR question-answering benchmarks, the 83.2 average beats Qwen3.5-2B (80.9) and trails Qwen3.5-4B (84.9). It's behind Qwen3.5-2B on OCRBench, 84.6 against 85.9. The 59.09% key-information score comes from Xiaomi's own 1,655-image set, and the report says the model trails Qwen3.5-2B on invoices (65.00% against 77.00%).

What you'd actually run

The checkpoint is Apache-2.0 on the model card, built on Qwen3.5-0.8B-Base. The card shows a whole-page prompt with Transformers, which skips the layout stage. The two-stage route is in the GitHub repository, and its Ollama guide says the optional region mode still needs PaddleX.

Xiaomi's own numbers show what the layout stage is worth. In its pull-request reply it lists 93.45 for whole-page inference and 95.24 for two-stage on Real5, both scored with the v1.6 code. That's 1.79 points, our subtraction.

The Ollama path is labeled experimental and uses a community BF16 package of about 1.8 GB. The repo's own guide says "accuracy parity is not established": 5 of 7 complete responses matched the Hugging Face BF16 run, and one dense-equation answer dropped a transpose. We found no latency or memory figures in the model card or install notes.

Is it free to use commercially? Apache-2.0 on the model card says yes for the weights; check the repository's code license before shipping.

Does the license cover the layout detector? Unconfirmed. PP-DocLayoutV3 is a separate model.

Will the leaderboard number change? Possibly. A member of the benchmark team said Real5 will be upgraded and new metrics issued; no date was given.

Run your own fifty pages twice

If you parse scans, photos or invoices, take fifty of your real pages and run them both ways: the model card's whole-page prompt, then the layout-first pipeline from the repository. Diff the two outputs on the fields you actually extract. On Xiaomi's numbers the layout stage is worth about 1.8 points, and the scoring code moved the headline by another 2.5. Your pages are the only benchmark that counts.

Source: Xiaomi-OCR-0 Technical Report (arXiv:2609.36136)

xiaomi xiaomi ocr 0 ocr open weights document parsing

AI

About the author

AI Briefing Daily Editorial. The editorial desk of AI Briefing Daily, an independent publication covering artificial intelligence. Every story is researched against primary sources, checked by hand and reviewed by a person before publication.