Verifiable Environments for AI in Biology — Kenny Workman, LatchBio
Summary
Kenny Workman discusses the massive data challenge in modern biology (e.g., single cell experiments yielding 2–6 terabytes per run) and proposes treating experimental biology as a 'verifiable substrate' for AI training. LatchBio is positioning itself as an agent engineering lab, building benchmarks like 'spatial bench' to train agents on scientific reasoning rather than just memorized knowledge. The core argument is that verifiable data analysis pipelines (modeled after code execution) are necessary because frontier models currently lack the capability to reliably extract scientific insight from complex, real-world biological datasets.
Key takeaways
-
Data Volume and Challenge
Modern techniques like single cell biology (2–6 TB per run) and spatial biology generate data volumes that exceed consumer storage capacity, necessitating automated analysis tools. The output is often a 'giant matrix of numbers' requiring complex processing.
-
Verifiable Substrate for Science
4:04
The goal is to treat biological data analysis as an executable substrate, similar to how code provides verifiability in software. This allows the creation of natural benchmarks and measurable progress metrics.
-
Benchmarking Scientific Reasoning
5:23
Benchmarks must move beyond simple Q&A settings to measure complex, multi-step scientific reasoning. LatchBio developed 'spatial bench' (146 problems) which requires deterministic graders and verifiable outcomes based on analysis DAGs.
-
Agent Limitations and Progress
9:00
Frontier models cannot be trusted with real-world scientific work because they are missing the capability between knowing biology and writing code. The process requires iterative human verification and developing tasks that force rigorous reasoning.
Technical details
-
Biological Data Types
0s
Key experimental classes include single cell biology (splitting cells to measure RNA), spatial biology (measuring RNA spread geometrically over tissue), and proteomics. Single cell experiments can yield 2–6 terabytes per run.
-
Spatial Biology Technique
One technique discussed is sequencing based spatial, which involves fusing little beads with DNA attached to them to capture RNA within a tissue section and determine its precise geometric resolution.
-
Benchmark Structure
Benchmarks are designed around an analysis DAG (Directed Acyclic Graph) structure. An evaluation requires: 1) one or more data nodes (e.g., a matrix of numbers/high-content image), 2) a task prompt describing the scientific goal, and 3) a deterministic grader (a Python function) to ensure verifiability.
-
Long Horizon Tasks
To simulate real-world research, benchmarks are extended to 'long horizon' tasks that require multiple steps and contextualization against prior literature or the original experimental design (e.g., reconstructing a metastatic niche).
Mentioned resources
- LatchBio
- spatial bench
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.