NVIDIA AICR for Dynamo: What 'Validated' Means on Kubernetes

NVIDIA says you can now stand up its Dynamo inference framework on Kubernetes from a single "recipe" that pins every component underneath it, then check the cluster against that recipe before any model goes on. The walkthrough, published Oct. 30, 2026 on the NVIDIA Technical Blog, is about NVIDIA AI Cluster Runtime, or AICR, and the word doing the work in its title is "validated."

It's worth reading that word closely. The post is specific about what was tested, and the specifics are narrower than the headline.

NVIDIA AI's post on X, dated Nov. 3, announcing the AICR walkthrough for Dynamo (@NVIDIAAI)

View on X

The post from @NVIDIAAI asks, "Have GPUs in Kubernetes?" and says AICR "brings together NVIDIA Dynamo and the infrastructure it needs, with tested component versions and checks to verify your setup." The code and recipes live in NVIDIA's open-source AICR repository and the Dynamo repository, both linked from NVIDIA's post. Below, the facts come from NVIDIA's two posts and are attributed. Where we're interpreting, we say so.

Seventeen pinned charts, one alpha

A Dynamo recipe is more than Dynamo. According to the post, it ships version 1.4.2 of the dynamo-platform chart, which includes the operator, the runtime images and the nvidia.com/v1beta1 DynamoGraphDeployment resource. That v1beta1 API replaces the earlier v1alpha1, so manifests written for the old version are the first thing to check.

Around it sit the pieces a GPU cluster needs: the NVIDIA GPU Operator, Node Feature Discovery, the DRA driver for GPUs, the KAI Scheduler, Grove for multi-component serving graphs, NVSentinel and Nodewright for node health, a Prometheus stack and agentgateway for ingress.

For the Amazon EKS recipe aimed at GB200 NVL72 and GB300 NVL72 systems, NVIDIA lists AICR v1.0.0 selecting 17 Helm chart versions. Among them: GPU Operator v26.7.1, DRA GPU Driver 0.5.0, KAI Scheduler v0.16.9, agentgateway v1.5.0 and Grove 0.1.0-alpha.13.

That last one stands out. A stack sold as tested and pinned includes a component whose own version string says alpha. Our reading: pinning is exactly the right answer to an alpha dependency, because it freezes the combination you tested. But it also means the "validated" label covers a snapshot of fast-moving parts, and the version you get today is the version you stay on until the recipe moves.

One default also changed. With Dynamo 1.4, the post says frontends, workers and the optional gateway components discover each other through Kubernetes resources, so the AICR path no longer installs NATS. It remains an opt-in.

Five commands before your first model

The runtime and the application are deliberately separate. You install the first, check it, and only then apply your own DynamoGraphDeployment.

In NVIDIA's walkthrough, aicr snapshot records the cluster. aicr recipe --intent inference --platform dynamo reads that snapshot and picks the Kubernetes distribution, GPU and node OS. aicr bundle renders Helm values and manifests, with --deployer helmfile, helm, argocd, argocd-helm or flux. Then you apply the bundle (helmfile diff, then helmfile apply) and run aicr validate.

One practical detail from the post: the placement flags in the bundle step, node selectors and tolerations, reflect NVIDIA's own test cluster. Yours will have different labels and taints. And if there's no default StorageClass, the bundle command needs --storage-class.

Two ways in, and one setting that can drift

Dynamo's headline feature for agent workloads is cache-aware routing. Agents resend long prefixes (system prompts, tool definitions, history), so a request that lands on a worker already holding those KV blocks skips recomputing them. The recipe supports two paths to get there:

• Dynamo Frontend router. The frontend scores workers on prefix overlap and load. NVIDIA calls it the smallest topology and "the most production-proven serving option."
• Gateway API plus Dynamo's endpoint picker. The gateway forwards traffic; Dynamo's picker chooses the worker, and each worker pod carries a frontend sidecar. NVIDIA calls it suitable when a platform team already owns shared ingress, timeouts and retries.

Both use cache events that workers publish, over ZMQ in Dynamo 1.4.2, while requests travel over TCP. The post also notes that Dynamo can instead predict cache state from where it sent each request. Prediction can't see evictions, so the router's picture can drift under memory pressure. The AICR configuration turns live events on. If you copy the routing pieces into a hand-built stack, that's a setting to carry over.

One roadmap line matters for the gateway path: starting with Dynamo 1.5, a native Rust endpoint picker is meant to replace today's Go-plus-CGO bridge. NVIDIA gives no date.

What "validated" does and doesn't promise

Here's what the sources support.

NVIDIA's AICR v1.0 post, published Oct. 6, says the project has more than 100 contributors, almost half from outside NVIDIA. It says v1.0 fixes compatibility rules for the CLI, REST API, Go SDK, bundle layout and artifact schemas, and that operators can inspect signed validation evidence for each recipe marked Supported: what was tested, which checks passed and who signed.

The Dynamo walkthrough's validation step covers deployment and conformance checks. Those include allocatable GPU capacity, a working nvidia-smi, GPU Operator and Dynamo health, DRA access, gang scheduling, autoscaling, Prometheus metrics and Gateway API readiness. Performance validation runs separately, because it deploys a temporary serving workload and uses the GPUs under test.

Our reading, clearly ours: this is a check that the plumbing matches the recipe, not a promise about speed or cost per token. The post's version table is for one environment, EKS on GB200 and GB300 NVL72. We can't tell from either post how the recipes for other clouds or GPU generations compare, and the v1.0 post says contributors can propose recipes for combinations the maintainers can't test. If you're on a different cloud or older GPUs, look up your exact combination and whether its evidence is published before treating "validated" as applying to you.

NVIDIA also says an offline --no-cluster validation against a different snapshot evaluates declarative constraints only, with behavioral checks reported as skipped. Our reading: a green offline run is a paper check.

Quick answers before you try it

Is AICR a Kubernetes operator? Not as NVIDIA describes it. The v1.0 post says a recipe describes desired configuration and "does not reconcile a cluster"; Helm, Argo CD, Flux or Helmfile do the applying.

What changes from earlier Dynamo setups? Per the walkthrough: the v1beta1 API replaces v1alpha1, and NATS is no longer installed by default.

Does it work on any GPU cluster? Unconfirmed. The only version table published is for EKS with GB200 and GB300 NVL72. Check the recipe and its evidence for your own combination.

Does a passing validation mean it will be fast? No. Performance validation is a separate, GPU-consuming run, and the post doesn't publish results for it.

The question to put to your own cluster

Before you run the snapshot, answer this: which of the 17 pinned versions do you already run, and which would your platform team have chosen differently? The recipe is only as useful as the gap you can see between it and your current stack. NVIDIA lists a hands-on AICR workshop at KubeCon North America (registration required) and a booth, #170, for anyone who wants to ask the maintainers directly.

Source: NVIDIA Technical Blog

nvidia dynamo aicr kubernetes inference

AI

About the author

AI Briefing Daily Editorial. The editorial desk of AI Briefing Daily, an independent publication covering artificial intelligence. Every story is researched against primary sources, checked by hand and reviewed by a person before publication.