Hugging Face Journal Club: Training AI Scientists to Replicate Research
Summary
The discussion summarizes research on Faraday-27B, a model trained by Inherent designed for scientific replication—the ability to reproduce results from redacted ML/AI papers. The system uses Reinforcement Learning (RL) and integrates CodeX as a tool, allowing the agent to execute code within a simulated environment. Key methodological advances include using sophisticated rubric-based judges (generated via Claude) instead of simple verifiers, employing multi-rollout averaging to mitigate variance, and implementing weighted credit assignment across the agent's steps.
Key takeaways
-
Scientific Replication Task
The model is tasked with replicating missing figures from redacted ML/AI papers. This process requires the agent to use tools (like CodeX) and execute code in a simulated environment, moving toward full automation of AI R&D.
-
Advanced Judging Mechanism
0:01
Instead of simple verification, the system uses a rubric-based judge (generated by Claude) that assigns fine-grained points for correct reasoning, figure accuracy, and code writing. This process involves averaging judgments across multiple rollouts to prevent reward hacking.
-
Performance & Scaling
0:02
The trained Faraday model demonstrated strong performance, sometimes outperforming much larger models like Claude and GPT-5. Furthermore, the system showed generalization even when given increased compute resources (e.g., scaling up to 8 hours/8 B300s).
Technical details
-
Training Methodology
1s
The model is trained using GPO (Generalized Policy Optimization) and LORA. The training utilizes a weighted credit assignment mechanism, which assigns higher reward weights to turns where the agent successfully uses CodeX as a tool, encouraging better tool utilization.
-
Infrastructure Stack
3s
The infrastructure for training is complex, utilizing an Nvidia-based job setup that incorporates Megatron and Nemo RL. The system manages resources using a Kubernetes cluster, with Ray handling the assignment of GPUs for both training and generation rollouts.
-
Model Architecture
4s
The agent (Faraday-27B) acts as an overseer, utilizing CodeX as a tool. The system prompt is highly detailed, guiding the model to act like a 'research coder' and follow scientific protocols.
Mentioned resources
- Faraday-27B Model
- Hugging Face Paper Link
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.