# Verifiable Environments for AI in Biology — Kenny Workman, LatchBio

## Executive summary

Kenny Workman discusses the massive data challenge in modern biology (e.g., single cell experiments yielding 2–6 terabytes per run) and proposes treating experimental biology as a 'verifiable substrate' for AI training. LatchBio is positioning itself as an agent engineering lab, building benchmarks like 'spatial bench' to train agents on scientific reasoning rather than just memorized knowledge. The core argument is that verifiable data analysis pipelines (modeled after code execution) are necessary because frontier models currently lack the capability to reliably extract scientific insight from complex, real-world biological datasets.

## Key takeaways

- Data Volume and Challenge: Modern techniques like single cell biology (2–6 TB per run) and spatial biology generate data volumes that exceed consumer storage capacity, necessitating automated analysis tools. The output is often a 'giant matrix of numbers' requiring complex processing.
- Verifiable Substrate for Science: The goal is to treat biological data analysis as an executable substrate, similar to how code provides verifiability in software. This allows the creation of natural benchmarks and measurable progress metrics.
- Benchmarking Scientific Reasoning: Benchmarks must move beyond simple Q&A settings to measure complex, multi-step scientific reasoning. LatchBio developed 'spatial bench' (146 problems) which requires deterministic graders and verifiable outcomes based on analysis DAGs.
- Agent Limitations and Progress: Frontier models cannot be trusted with real-world scientific work because they are missing the capability between knowing biology and writing code. The process requires iterative human verification and developing tasks that force rigorous reasoning.

## Technical details

- Biological Data Types: Key experimental classes include single cell biology (splitting cells to measure RNA), spatial biology (measuring RNA spread geometrically over tissue), and proteomics. Single cell experiments can yield 2–6 terabytes per run.
- Spatial Biology Technique: One technique discussed is sequencing based spatial, which involves fusing little beads with DNA attached to them to capture RNA within a tissue section and determine its precise geometric resolution.
- Benchmark Structure: Benchmarks are designed around an analysis DAG (Directed Acyclic Graph) structure. An evaluation requires: 1) one or more data nodes (e.g., a matrix of numbers/high-content image), 2) a task prompt describing the scientific goal, and 3) a deterministic grader (a Python function) to ensure verifiability.
- Long Horizon Tasks: To simulate real-world research, benchmarks are extended to 'long horizon' tasks that require multiple steps and contextualization against prior literature or the original experimental design (e.g., reconstructing a metastatic niche).

## Practical implications

- The development of verifiable benchmarks (like 'spatial bench') provides a measurable way to advance AI capabilities in complex scientific domains.
- Agentic systems are being built to handle multi-day or multi-week data analysis operations, moving beyond simple chat interfaces.
- Biosecurity tasks involve developing red team evaluations that test for misuse potential (e.g., cloning genes into bacteria) alongside routine scientific queries.

## Topics

AI Agent Engineering, Spatial Biology, Scientific Benchmarking, Data Analysis Pipelines, Large Language Models (LLMs), LatchBio, spatial bench

Source: https://www.youtube.com/watch?v=3ZMUiFaQ3qg