ToolsCompaniesAbout

NVIDIA Dynamo

by NVIDIA@nvidiaSanta Clara, California, USANo ratings yet

Open-source framework for serving LLMs across multi-GPU, multi-node clusters

Visit nvidia.com
Tool type
Distributed LLM inference framework
Made for
Infrastructure and ML platform engineers
Runs on
Self-hosted GPU clusters
Docker containers, PyPI install, or Kubernetes; NVIDIA GPUs expected
Last checked
28 Oct 2026
3 product sources

What it does

NVIDIA Dynamo is an open-source inference-serving framework that sits above engines such as SGLang, TensorRT-LLM and vLLM. You give it a model and a GPU cluster, and it returns an OpenAI-compatible serving endpoint that coordinates many GPUs or nodes. It splits prefill and decode onto separate GPU pools, routes requests to avoid repeated computation, and offloads KV cache from GPU memory to CPU RAM, SSD or network storage.

Teams typically start from a prebuilt container or PyPI package for local testing, then move to Kubernetes for production using the Dynamo Platform. Extra components include a planner that scales GPU pools to meet latency targets, Grove for topology-aware Kubernetes scheduling, AIConfigurator for suggesting prefill/decode settings, and AIPerf for benchmarking. It coordinates inference engines rather than replacing them.

It suits infrastructure and ML platform engineers running large models across several GPUs or nodes. Using it requires NVIDIA GPUs, container or Kubernetes skills, and one of the supported backends. For a single model on a single GPU, the project says the inference engine alone is probably enough. The source is free to download, but GPU hardware and operations are your own cost. Some features are marked beta or in progress.

What it can do

  • Disaggregated prefill and decode serving on separate GPU pools
  • LLM-aware, KV-cache-aware request routing
  • KV cache offload from GPU to CPU RAM, SSD or remote storage
  • SLA-driven Planner that autoscales GPU workers
  • Topology-aware Kubernetes scheduling with Grove
  • Benchmarking with AIPerf and configuration advice via AIConfigurator

Works with

  • SGLang
  • NVIDIA TensorRT-LLM
  • vLLM
  • Kubernetes
  • LMCache
  • LangChain
  • NVIDIA NeMo Agent Toolkit

Example workflows

Try a model locally in a container
Input: a GPU machine with Docker and a small model such as Qwen3-0.6B. Steps: pull the vLLM or SGLang runtime container, start the Dynamo frontend and a worker, then send a chat request to the local endpoint. Output: a working OpenAI-compatible endpoint for testing.
Deploy a model to Kubernetes with latency targets
Input: a Kubernetes cluster with Dynamo Platform installed, a model name, backend and latency targets (TTFT and ITL). Steps: write a DynamoGraphDeploymentRequest manifest and apply it. Output: a deployment sized and optimized by Dynamo against those targets (feature marked beta).

Before you choose

  • It coordinates inference engines rather than replacing them, so you still choose and operate a backend and the GPU hardware
  • Some features are beta or in progress (for example zero-config deploy and KVBM on some backends); check the feature matrix before relying on them
  • Enterprise support is tied to NVIDIA AI Enterprise; request a quote from NVIDIA for support terms

Listed under

Get started

Pull a container or install from PyPI; Kubernetes recommended for production

  1. Review the GitHub README and support matrix to confirm your GPUs, OS and backend are supported
  2. Run a prebuilt vLLM, SGLang or TensorRT-LLM runtime container, or install the ai-dynamo package
  3. Start the frontend and a worker, then test with a chat completions request; move to Kubernetes for multi-node use
Get it on GitHub

Plans and prices

The source code is available under Apache 2.0. GPU infrastructure and optional enterprise support are separate.

Open-source softwareFreeCode only; excludes infrastructure and external services
  • Apache 2.0 license

Source: github.com/ai-dynamo/dynamo/blob/main/LICENSE, checked 28 Oct 2026. Prices can change; check before you buy.

Sources for this overview

Product pages, documentation, and pricing checked 28 Oct 2026. Each source supports part of the overview; availability and prices can change.

Ratings and reviews

No ratings for NVIDIA Dynamo yet.

Be the first to rate it

What’s changed

Free account

See what’s changed at NVIDIA Dynamo

Launches, price moves and pivots, each with a date and a source. Free with an account.

Signals

Free account

See NVIDIA Dynamo’s signals

Launch traction, hiring and release activity, tracked over time. Free with an account.

Data and security

Free account

See how NVIDIA Dynamo handles your data

Training on your files, compliance, hosting and the models behind it. Free with an account.

Compare with similar tools

Free account

Compare NVIDIA Dynamo side by side

Prices, platforms and features against similar tools, with sources. Free with an account.

Where this comes from

How we research
Company website
nvidia.com: product, pricing and policy pages
YC directory mirror
Batch and company name
Web search
Confirms the official site
Social accounts
Accounts the company links, and its posts on X
Human review
Checked by a person
knowledge_base

Last checked 28 Oct 2026. Every field keeps its source and date; unknown means we couldn’t confirm it.