# NVIDIA Dynamo in 5 Minutes: Dynamo On Kubernetes for AI Inference

## Executive summary

This video details how to deploy NVIDIA Dynamo for scalable, production-grade AI inference on Kubernetes. Dynamo utilizes the operator pattern to manage the serving setup, allowing users to define their desired state (via DGDR or DGD) and letting the operator handle the complex reconciliation into running Kubernetes resources. The architecture ensures a stable, OpenAI-compatible API front end, which routes requests to specialized backend workers (like SGLang, vLLM, or TensorRT-LLM), decoupling the client interface from the evolving serving platform.

## Key takeaways

- The Problem with Single Model Servers: Single model servers are easy for demos but fail to meet production requirements for scalability, standardized deployment, and service management.
- Dynamo Operator Functionality: Dynamo uses the operator pattern on Kubernetes: the user defines the desired state (the deployment request), Kubernetes stores it, and the operator works to make the actual cluster match that desired state.
- Two Deployment Paths: Users can choose between the intent-based Dynamo Graph Deployment Request (DGDR) or the predefined Dynamo Graph Deployment (DGD) path, depending on whether they are starting from scratch or using a known good recipe.
- Request Flow and Decoupling: A client interacts with a stable, OpenAI-compatible front end. Dynamo routes requests to backend workers, which run the model engine (e.g., vLLM, SGLang). This separation allows the serving platform to evolve without changing the client API.

## Technical details

- Dynamo Operator Pattern: The operator pattern allows the team to define the desired serving setup, which the operator then manages and reconciles into the necessary Kubernetes resources. This shifts deployment from custom scripts to a repeatable Kubernetes workflow.
- Deployment Paths (DGDR vs. DGD): The DGDR is the intent-based path where the user describes the model, backend, workload, hardware, and optional SLA targets. This profiles into a DGD. Alternatively, the DGD path starts with a known good deployment recipe and allows adjustment of the inference graph and serving topology.
- Request Routing and Engines: The client sends requests to the OpenAI-compatible front end. Dynamo routes these requests to backend workers, which execute the model using specific engines such as SGLang, vLLM, or TensorRT-LLM, depending on the configuration.

## Practical implications

- Automates the complex process of deploying AI models by abstracting away service discovery, scaling, and routing into a Kubernetes-native workflow.
- Provides a stable, familiar OpenAI-compatible API boundary, insulating the client from changes in the underlying model engine or serving architecture.
- Reduces operational overhead by replacing custom deployment scripts with declarative Kubernetes manifests and operators.

## Topics

Kubernetes, AI Inference, Operator Pattern, Distributed Systems, OpenAI API Compatibility, Model Serving, NVIDIA Dynamo Documentation, Dynamo GitHub Repository

Source: https://www.youtube.com/watch?v=JnsYIojzM5I