Verifiable Environments for AI in Biology — Kenny Workman, LatchBio
Kenny Workman discusses the massive data challenge in modern biology (e.g., single cell experiments yielding 2–6 terabytes per run) and proposes treating experimental biology as a 'verifiable substrate' for AI training. LatchBio is positioning itself as an agent engineering lab, building benchmarks like 'spatial bench' to train agents on scientific reasoning rather than just memorized knowledge. The core argument is that verifiable data analysis pipelines (modeled after code execution) are necessary because frontier models currently lack the capability to reliably extract scientific insight from complex, real-world biological datasets.
Key takeaways
-
Data Volume and Challenge
Modern techniques like single cell biology (2–6 TB per run) and spatial biology generate data volumes that exceed consumer storage capacity, necessitating automated analysis tools. The output is often a 'giant matrix of numbers' requiring complex processing.
-
Verifiable Substrate for Science
4:04
The goal is to treat biological data analysis as an executable substrate, similar to how code provides verifiability in software. This allows the creation of natural benchmarks and measurable progress metrics.
-
Benchmarking Scientific Reasoning
5:23
Benchmarks must move beyond simple Q&A settings to measure complex, multi-step scientific reasoning. LatchBio developed 'spatial bench' (146 problems) which requires deterministic graders and verifiable outcomes based on analysis DAGs.
-
Agent Limitations and Progress
9:00
Frontier models cannot be trusted with real-world scientific work because they are missing the capability between knowing biology and writing code. The process requires iterative human verification and developing tasks that force rigorous reasoning.