Cost Optimization

Aug 10, 2026 · 17:16

Prompt Caching Explained: Stop Overpaying for AI Agents

Prompt caching is essential for managing costs in long-running AI agent sessions. Instead of paying full price for re-sending entire conversation histories (context windows) on every turn—which can lead to exponential cost increases—proper prompt caching ensures that the LLM only charges a discounted rate for tokens it has seen before. This requires designing an agent harness that correctly preserves reusable prompt prefixes and understands provider-specific API behaviors.

Key takeaways

1. Cache Inputs, Not Outputs

   Prompt caching stores and reuses inputs (the conversation history/prompt), not the LLM's outputs. Caching outputs is generally not useful for LLMs.

2. Cost Escalation Risk 0:23

   Without caching, sending a growing context window (e.g., 51k tokens, then 55k tokens) repeatedly leads to exponentially increasing costs, making long sessions prohibitively expensive.

3. Cache Expiry is Critical 2:03

   The cache has an expiration time (e.g., OpenAI OAuth API: one hour; Anthropic: five minutes by default). The agent harness must account for this expiry to avoid paying full price again.

4. System Prompts Must Be Static 8:00

   To prevent cache invalidation, do not include dynamic elements like timestamps or current working directories within the system prompt. Keep conversation history append-only.

• Hugging Face
• AI Agents
• LLM Architecture
• Cost Optimization
• Prompt Engineering
• Software Design
• Written tutorial: Prompt Caching
• Tau coding agent
• Pi coding agent
• Hugging Face Inference Providers
• OpenAI prompt caching
• Anthropic prompt caching
• Gemini context caching

Watch on YouTube Full article

Jun 29, 2026 · 26:20

GLM-5.2 vs MiniMax-M3: Opus Has REAL COMPETITION (Model Stacking)

The video argues that proprietary models like Opus 4.8 face real competition from open-weight alternatives such as GLM-5.2 and MiniMax-M3. The core thesis for build engineers is not to select a single model but to implement a resilient 'model stack.' This strategy involves strategically choosing models across three tiers—State-of-the-Art (SOTA), Workhorse, and Lightweight/Local—to optimize the trade-off between performance, cost, and speed for both engineering agents and product deployment.

Key takeaways

1. GLM 5.2 vs MiniMax M3: Performance vs Cost 17:54

   GLM 5.2 is highlighted as the better model in terms of raw performance (A-tier), while MiniMax M3 is considered the better deal due to its optimized cost structure, making it ideal for high-volume product agents.

2. The Three-Tier Model Stack Framework 2:50

   Engineers should categorize models into three tiers: State-of-the-Art (e.g., Opus 4.8, Fable 5), Workhorse (GLM 5.2, MiniMax M3), and Lightweight/Local (Qwen 3.6). This framework guides decision-making based on the required trade-off.

3. Resilience through Open Weights 6:49

   Due to concerns about vendor lock-in or potential service shutdowns (e.g., Fable), relying solely on closed-source models is risky. Utilizing open-weight models like GLM 5.2 and MiniMax M3 ensures greater control and ownership over the AI infrastructure.

• IndyDevDan
• Large Language Models (LLMs)
• Agentic Engineering
• Model Stacking
• Open-Source AI
• Cost Optimization
• AI Infrastructure Architecture
• AA Article — GLM-5.2 is the new leading open-weights model...
• AA Article — MiniMax-M3:
• AA — Our 4-model comparison (intelligence vs tokens, open vs proprietary...)

Watch on YouTube Full article