Why LLM Recommenders Will Be AI's Biggest Consumer App — Devansh Tandon, Meta
Summary
Devansh Tandon argues that recommendation systems (recsys) are following scaling laws similar to Large Language Models (LLMs), positioning the LLM recommender as one of the most significant and token-efficient consumer AI applications. The core technical advancements involve using Semantic IDs to compress content tokens and building a multi-stage LLM recommender (pre-training, post-training) that can reason over user history and content. This shift enables 'steerable' feeds, giving users control over their algorithmic experience.
Key takeaways
-
LLM Recommenders are a Major Consumer AI Application
The LLM recommender is predicted to be one of the biggest consumer applications of AI because content feeds are structurally more token-efficient than general AI chat apps, making them up to 100x cheaper to run for an hour of engagement.
-
Recommendation Systems Follow Scaling Laws
3:35
Recsys follow a power law of scaling similar to LLMs, where increasing model size, data, and compute leads to predictable improvements in recommendation quality and real-world engagement impact.
-
The LLM Recommender Recipe
13:25
Building an LLM recommender involves three steps: 1) Tokenizing content using Semantic IDs, 2) Adapting the LLM to be bilingual (English and domain language), and 3) Post-training the model to rank content using chain-of-thought reasoning.
Technical details
-
Semantic IDs and Tokenization
935s
Semantic IDs are used to convert long content sequences (e.g., a three-minute Reel) from thousands of tokens into a highly compressed, stable representation (e.g., 10 tokens), enabling the LLM to reason over content context windows efficiently.
-
LLM Recommender Architecture
The process is described as a five-layer cake: Semantic ID layer $\rightarrow$ Base LLM $\rightarrow$ Pre-training (bridging English and recommender tokens) $\rightarrow$ Post-training (steering for tasks like predicting engagement) $\rightarrow$ Surface-specific fine-tuning.
-
Steerable Feeds
By training the model to understand both English and recommendations, users can interact with the algorithm in natural language (e.g., 'I want to follow the FIFA World Cup'), allowing them to guide the content feed's output.
-
Flywheel Dynamics
455s
The 'tokens in, engagement out' flywheel describes the cycle: Model Training $\rightarrow$ Inference (Tokens In) $\rightarrow$ Consumer Engagement $\rightarrow$ Monetization (Paying for next Model Training).
Mentioned resources
- Meta AI
- Meta Recommendations Research
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.