# 40 Trillion Tokens a Day (Yes, More Than OpenAI) | Lin Qiao, CEO of Fireworks

## Executive summary

The discussion centers on the industry shift from general-purpose AI models (like those from OpenAI/Anthropic) toward specialized intelligence. Lin Qiao of Fireworks argues that true innovation lies in leveraging proprietary, locked-in enterprise data—the 'alpha'—to build customized models. She asserts that this specialization is necessary because generalized models cannot capture a company's unique knowledge or judgment. Technically, the conversation details advanced training methods (SFT, DPO, KTO, RL) and emphasizes platform control, noting that Fireworks achieves bitwise equivalence between training and inference results to ensure maximum quality while optimizing for cost and speed.

## Key takeaways

- The Rise of Specialized Intelligence: Lin Qiao argues that the future belongs to specialized intelligence—customized models built on private company data—rather than general-purpose AGI. She believes every company is unique, making it difficult for a single general model to capture proprietary knowledge (41:35).
- Fireworks' Scale and Focus: Fireworks claims to process over 40 trillion tokens daily, stating that 95% of this traffic comes from customized model inference deployment, not off-the-shelf APIs. This volume surpasses both OpenAI API and Gemini API usage (13:36).
- Open vs. Closed Models for Security: Lin Qiao suggests that open models are better suited to strike a balance in the security debate, encouraging broader community participation to increase defensive complexity against potential cyber threats (47:00).

## Technical details

- Model Customization and Training Paradigms: The process of specialization must be continuous. Techniques discussed include Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), KTO, and Reinforcement Learning (RL). RL is highlighted as a method where the model learns by interacting with a product or simulation environment and receiving a 'reward' signal, rather than just being shown ground truth (23:36).
- Platform Engineering & Optimization: Fireworks emphasizes its proprietary platform design to maximize quality while optimizing for speed and cost. A key technical achievement is reaching bitwise equivalence between training and inference results, which is crucial for maintaining precision during deployment (50:51).
- Self-Serve Training Infrastructure: Fireworks offers a self-serve training SDK that allows developers to plug in custom loss functions and tweak algorithms. This enables customers with varying levels of AI expertise to conduct complex experiments without needing full expert intervention (28:53).

## Practical implications

- For build engineers, the shift mandates a focus on unit economics and cost control (value maxing) over simple token volume maximization. Solutions must integrate proprietary data into the model lifecycle to justify expenditure.
- The need for specialized intelligence means that infrastructure design must support continuous, highly customized inference deployment rather than relying solely on general API calls.
- Building robust internal evaluation functions (evals) is critical, as product signals and domain-specific knowledge are required to define effective reward functions for RL fine-tuning.

## Topics

AI Infrastructure, LLM Deployment, Model Fine-Tuning, Unit Economics, Proprietary Data Strategy, Fireworks, Weights and Biases (W&B), Lin Qiao

Source: https://www.youtube.com/watch?v=Tf7DEYfZc0g