# Specializing AI for Regulated Industries - How Domyn Uses NVIDIA Nemotron

## Executive summary

This livestream details Domyn's journey toward building a family of sovereign AI models for regulated industries, emphasizing full ownership and control over the entire model stack. The presentation covers advanced LLM development techniques—including model compression (pruning/distillation), continual pretraining (CPT), Supervised Fine-Tuning (SFT), and Reinforcement Learning (RL)—using the NVIDIA open source ecosystem. A significant focus is placed on the tooling required to manage these complex pipelines at scale, including custom tools like Swarm and FinalFlows, which are crucial for build engineers managing large-scale compute infrastructure.

## Key takeaways

- Sovereign AI requires full stack control: For regulated industries, achieving sovereign AI necessitates controlling every layer of the stack—from compute to model ownership—rather than relying on point solutions. This approach is critical for governance and auditability. (0:28-1:35)
- Domain Large development pipeline: Domyn developed Domain Large by starting with Coliseum 355, followed by model compression (pruning/distillation), CPT to expand context up to 128K tokens, and SFT to enable reasoning. This was executed using the NVIDIA stack on H200s in DJX cloud. (4:36-7:39)
- Domain Small for efficiency: To address cost concerns associated with large models, Domain Small (10B parameters) was created. Its training heavily leveraged Reinforcement Learning (RL) and Direct Preference Optimization (DPO), demonstrating that smaller models can achieve strong performance through advanced post-training curricula. (14:23-17:56)
- Tooling for scalable ML pipelines: Domyn developed internal tools like Swarm (CLI/Python) and FinalFlows (DAG library) to manage complex, interconnected jobs on Slurm clusters. These tools remove friction when running large-scale evaluations and training across European infrastructure. (24:08-31:56)

## Technical details

- Model Compression & Optimization: The process involved pruning and distillation, with the finding that cutting only ~10% of depth/width was sufficient to recover almost all model performance. The loss function used included cross-entropy regional one and KL loss acting as a regularizer. (6:38-7:29)
- Training Methodologies: The development utilized advanced techniques including Continual Pretraining (CPT) up to 128K tokens, Supervised Fine-Tuning (SFT), and Reinforcement Learning with verifiable rewards. DPO was used for post-training optimization on Domain Small. (7:39-10:45; 16:00-17:56)
- Agentic Capabilities & Tool Use: The platform allows models to act as agentic systems, demonstrated by Domain Large generating complex SQL queries based on a provided schema. The architecture supports integrating external tools (e.g., web search) into the model workflow. (12:06-13:45; 29:28-30:47)
- Compute Infrastructure: The models were trained on dozens of H200s in DJX cloud, and the final frontier model development is leveraging European compute clusters designed for scientific computing. (7:59; 38:40-39:30)
- MLOps Tooling: Domyn developed Swarm (CLI/Python) to manage inference replicas and load balancing on Slurm, Domino for aggregating evaluations across multiple benchmark suites, and FinalFlows (a DAG library) for managing interconnected jobs with built-in fault tolerance. (24:08-31:56)

## Practical implications

- Build engineers managing LLM deployments should prioritize developing robust MLOps tooling (like DAG management and load balancing) to handle the complexity of multi-stage training and evaluation pipelines.
- The trend toward smaller, specialized models (e.g., Domain Small) suggests that optimizing for efficiency and cost via advanced post-training techniques (DPO/RL) is critical for enterprise deployment in regulated environments.
- Implementing provenance and traceability features—mapping model outputs back to source facts or knowledge graph entries—is essential for achieving verifiability required by highly regulated sectors.

## Topics

AI Model Development, LLM Architecture, MLOps, Regulated Industries, Compute Optimization, NVIDIA Nemotron, Domain Large, Domain Small, Megatron LM / Megatron Bridge, Azure Foundry, Hugging Face

Source: https://www.youtube.com/watch?v=1wfGCEzFBrk