# Hugging Face Journal Club: Training AI Scientists to Replicate Research

## Executive summary

The discussion summarizes research on Faraday-27B, a model trained by Inherent designed for scientific replication—the ability to reproduce results from redacted ML/AI papers. The system uses Reinforcement Learning (RL) and integrates CodeX as a tool, allowing the agent to execute code within a simulated environment. Key methodological advances include using sophisticated rubric-based judges (generated via Claude) instead of simple verifiers, employing multi-rollout averaging to mitigate variance, and implementing weighted credit assignment across the agent's steps.

## Key takeaways

- Scientific Replication Task: The model is tasked with replicating missing figures from redacted ML/AI papers. This process requires the agent to use tools (like CodeX) and execute code in a simulated environment, moving toward full automation of AI R&D.
- Advanced Judging Mechanism: Instead of simple verification, the system uses a rubric-based judge (generated by Claude) that assigns fine-grained points for correct reasoning, figure accuracy, and code writing. This process involves averaging judgments across multiple rollouts to prevent reward hacking.
- Performance & Scaling: The trained Faraday model demonstrated strong performance, sometimes outperforming much larger models like Claude and GPT-5. Furthermore, the system showed generalization even when given increased compute resources (e.g., scaling up to 8 hours/8 B300s).

## Technical details

- Training Methodology: The model is trained using GPO (Generalized Policy Optimization) and LORA. The training utilizes a weighted credit assignment mechanism, which assigns higher reward weights to turns where the agent successfully uses CodeX as a tool, encouraging better tool utilization.
- Infrastructure Stack: The infrastructure for training is complex, utilizing an Nvidia-based job setup that incorporates Megatron and Nemo RL. The system manages resources using a Kubernetes cluster, with Ray handling the assignment of GPUs for both training and generation rollouts.
- Model Architecture: The agent (Faraday-27B) acts as an overseer, utilizing CodeX as a tool. The system prompt is highly detailed, guiding the model to act like a 'research coder' and follow scientific protocols.

## Practical implications

- The research demonstrates a path toward automating complex scientific reasoning and experimental design within AI R&D.
- The combination of specialized models (Faraday) with powerful tools (CodeX) suggests future agents can perform highly structured, multi-step tasks that require both planning and execution.

## Topics

AI Research, Reinforcement Learning (RL), Large Language Models (LLMs), Tool Use, Scientific Computing, Faraday-27B Model, Hugging Face Paper Link

Source: https://www.youtube.com/watch?v=HxahKqN1q2g