# Podcast Crossover: AIE, AGI, frontier lab strategy with ​ ⁨@matthew_berman⁩ and @swyxtv

## Executive summary

The discussion explores the current state and future architectural challenges of frontier AI models. Key technical points covered include specialized hardware (e.g., Etched) optimizing for post-transformer workloads, the limitations of Large Language Models (LLMs) in achieving true recursive self-improvement (RSI), and the necessity for 'Agent Labs' to build model-agnostic applications that solve complex, last-mile problems.

## Key takeaways

- The Value Proposition of AI Engineering Conferences: AI conferences are becoming crucial neutral grounds where multiple frontier labs (like OpenAI) can compete on an even playing field, which is highly beneficial for engineers and competitive for the labs themselves. This contrasts with single-vendor events.
- Hardware Specialization vs. General Purpose AI: New generation chips (like Etched) are optimizing specifically for post-transformer workloads and architectures (post RGBT), moving beyond the general focus of older specialized hardware like Cerebras.
- Architectural Limitations of LLMs: LLMs are limited in their recursion because they tend to explore variations within known data distributions. True innovation and discovering 'unknown unknowns' still require dedicated research, suggesting a need for new architectural paradigms beyond current transformer models.
- The Future of Application Development: Founders should focus on building 'Agent Labs'—being the AI layer for specific industries (e.g., lawyers, dentists). This strategy is resilient to model generalization and capability overhangs because it solves persistent, last-mile problems.

## Technical details

- AI Hardware Architectures: New chips (e.g., Etched) are designed to optimize for post-transformer and post RGBT workloads, making them distinct from older specialized hardware like Cerebras.
- Model Efficiency and Learning Paradigms: Current LLMs are highly inefficient compared to human learning (which occurs over millions of data points). The next major breakthrough requires moving beyond the pre-train/post-train paradigm toward continual learning, potentially using only a few examples (e.g., 20 examples) to achieve agentic world modeling.
- Model Capabilities and Deployment: The concept of 'capability overhang' remains critical; the value extracted from current models (like Opus or GPT-5.5) is still high, ensuring that building tools around AI engineering remains a necessary function.
- Model Performance and Latency: Advanced models like Fable are noted for being extremely smart but also very slow. This slowness, coupled with high token costs, dictates that they should only be used for complex, 'smart' problems, not general tasks.

## Practical implications

- For founders, focus on becoming the 'AI layer' for a specific industry (e.g., legal tech) rather than betting on general model capabilities.
- Architects must design systems to be highly model-agnostic and capable of sophisticated model routing to maximize utility across different providers.
- The current trend suggests that while frontier labs push capability boundaries, the most sustainable business value lies in solving specific 'last mile' user problems using AI agents.

## Topics

AI Engineering, LLM Architecture, Agentic Systems, Hardware Acceleration, Model Agnosticism, Capability Overhang, Etched, Fable, Laten.space/agentlabs

Source: https://www.youtube.com/watch?v=bIOCKXcxloQ