# An Eval for AI Scientists That Knows the Answer Key - Ian Quigley, Leash Bio

## Executive summary

Leash Biosciences introduced Minos, a framework enabling scientific agents (like LLMs) to explore pre-measured, high-throughput chemical environments. This platform allows for the simulation of complex tasks, such as Structure-Activity Relationship (SAR) optimization in medicinal chemistry, by giving agents a limited experimental budget and requiring them to hypothesize and test their findings. The system functions as a 'lab-in-the-loop' with amortized latency, allowing for massive, counterfactually dense rollouts for evaluating scientific decision-making at scale.

## Key takeaways

- Agent Performance and Scientific Reasoning: More performant agents are significantly better at identifying and abandoning unproductive chemical neighborhoods, a key indicator of advanced scientific decision-making. Conversely, less capable models tend to persist in low-scoring areas, wasting their limited experimental budget. (00:04:30-00:05:10)
- Platform Scalability and Scope: The Minos approach allows for screening thousands of proteins against millions of molecules using pre-measured environments. The goal is to scale this capability, aiming to generate up to 20 new environments per day. (00:08:50-00:09:20)
- LLM Capabilities in Chemistry: LLMs are being used to perform complex chemical tasks, such as converting between different representations of molecules (e.g., canonical and non-canonical SMILES), which was previously difficult for frontier models. (00:09:40-00:10:10)

## Technical details

- Structure-Activity Relationship (SAR) Optimization: The core problem addressed is optimizing molecules to bind better to a target protein. Agents are given a catalog and a limited budget (e.g., 100 molecules) and must generate hypotheses about molecular improvements, which are then tested within the pre-measured environment. (00:02:20-00:03:10)
- Minos Architecture (Lab-in-the-Loop): The system is designed to be a 'lab-in-the-loop' with amortized latency. By pre-measuring the environment, it allows for a huge number of rollouts, making the process scalable and counterfactually dense. (00:07:30-00:08:00)
- Molecular Representation and Tool Use: The platform requires LLMs to handle chemical language, specifically SMILES (Simplified Molecular-Input Line-Entry System), which is a string representation of a molecule's graph traversal. The goal is to enable models to internalize the physical nature of molecules and use structural tool calls. (00:09:40-00:10:30)

## Practical implications

- Accelerating drug discovery by simulating complex SAR campaigns with limited physical resources.
- Developing general scientific decision-making skills in agents, applicable beyond medicinal chemistry.
- Creating scalable, high-fidelity RL environments for training AI agents on physical scientific processes.

## Topics

Artificial Intelligence, Reinforcement Learning, Medicinal Chemistry, Computational Biology, High-Throughput Screening, Minos Paper/Preprint, Blog post, Data and transcripts

Source: https://www.youtube.com/watch?v=vhPCo-8K78U