AI Can Now Predict 85%+ of What You’ll Do Next | Simile Co-founder & CEO, Joon Sung Park
Summary
Simile, co-founded by Joon Sung Park, presents a new paradigm in generative AI: moving beyond reasoning machines (like ChatGPT or Claude) to creating agents that accurately simulate human behavior. The company trains a foundation model of human behavior on data from tens of thousands of real people. Their methodology allows for predicting human actions with up to 85%+ accuracy, particularly in complex behavioral studies and randomized control trials (RCTs). This capability is valuable for product testing, understanding social dynamics, and simulating corporate decision-making.
Key takeaways
-
High Accuracy in Behavioral Prediction
0:17
The model can predict human behavior with 85%+ accuracy, especially when validating against real-world behavioral studies (e.g., Big Five personality tests, Prisoner's Dilemma). This accuracy is measured by correlation (Pearson correlation > 0.6 is robust; 0.75–0.8 is very strong).
-
Focus on Distribution, Not Reasoning
0:23
Unlike frontier models that focus on replicating expert reasoning (e.g., programming, math), Simile's model is designed to learn the accurate distribution of human values and behaviors across entire populations.
-
Simulation of Interventions (RCTs)
0:13
The most powerful application is simulating Randomized Control Trials (RCTs)—the ability to predict the outcome or effect of different interventions, which is critical for designing better social systems and products.
-
Model Architecture: Individual vs. Population
0:31
Simile trains two types of models: the 'model of individuals' (predicting what a specific person would do) and the 'population level model' (predicting the distribution of behavior across a group).
Technical details
-
Generative Agent Simulation
11s
The initial work involved creating a small game town with 25 agents using GPT-3.5, where agents exhibited emergent behaviors (e.g., self-organizing parties) without explicit planning.
-
Data Collection and Ground Truth
45s
Data is collected through partnerships (e.g., Gallup) with tens of thousands of real people across demographics. This data is used to train the foundation model, focusing on 'quantum units' like color preference or style, not just end outcomes.
-
Evaluation Metrics
17s
Validation involves running behavioral studies (e.g., Big Five, Prisoner's Dilemma) and measuring the correlation between agent predictions and real human responses. A Pearson correlation of 0.75–0.8 is considered very strong.
-
Model Training
31s
The models undergo fine-tuning (mid and post-training) using open-source models as a starting point, with significant effort dedicated to shaping parameters to represent people at scale.
Mentioned resources
- Simile
- Weights and Biases
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.