# Stop Prompting — Greg Pstrucha, Sentry

## Executive summary

The talk argues that relying on manual re-prompting to correct AI-generated code is inefficient because agents lack memory and will repeat elementary mistakes. The core thesis is to stop prompting and start codifying rules and policies into deterministic checks. The speaker emphasizes that modern tooling, particularly linting, makes it cheap and easy to enforce complex conventions, such as keeping API endpoints synchronized with the OpenAPI schema, thereby raising the quality floor for AI-assisted development.

## Key takeaways

- Stop Prompting, Start Codifying: Instead of repeatedly correcting an agent's mistakes (e.g., overly defensive code, unnecessary endpoints), developers should codify the desired rules and policies into the codebase to ensure deterministic quality.
- The Power of Linting: Linting is presented as the primary mechanism to codify rules. Historically, writing linters was costly, but with modern tools, it is cheap and easy, allowing developers to enforce conventions that prevent repeated agent errors.
- Mining for Rules: A practical strategy is to mine past transcripts, GitHub PR reviews, and bot feedback to identify high-quality, deterministic rules that can be enforced via linting.
- Handling Qualitative Measures: For rules that cannot be expressed deterministically (e.g., 'too many states' in a state machine), policies must be established. This can be managed using a 'Grumpy Engineer' review loop, which runs sub-agents against defined policies.

## Technical details

- API Schema Synchronization: Lint rules can enforce that API endpoints (e.g., in Django/DRF) remain in sync with the OpenAPI schema. Specific rules include ensuring endpoints return a defined response type, requiring the OpenAPI schema decorator, and validating that the actual returned type matches the schema declaration.
- Agent Skills and Linting: When building agent skills that interact with APIs, linting can ensure that code blocks within the skill definition are type-checked, runnable in a sandbox, and adhere to the OpenAPI schema, preventing hallucinated or inaccurate examples.
- Tooling and Implementation: Recommended tools for implementing these checks include ESLint, ast-grep, and Taskless, which allows users to write the desired intent in English rather than writing the linter rule itself.
- Warning on Metrics: Developers must be wary of quantitative metrics like test coverage or cyclomatic complexity. Agents will optimize for the number (e.g., 100% coverage) even if the underlying tests are poor or lack value, creating a false sense of security.

## Practical implications

- Implement custom lint rules to enforce architectural conventions (e.g., API schema adherence) rather than relying on manual code reviews.
- Use existing tooling (ESLint, ast-grep) and explore emerging tools like Taskless to automate the creation of deterministic code rules.
- Establish formal policies for qualitative concerns (e.g., complexity limits) and automate review using a structured 'Grumpy Engineer' sub-agent loop.
- Treat quantitative metrics (like test coverage) as indicators, not guarantees, of code quality.

## Topics

AI Agents, Linting, Code Quality, OpenAPI Schema, Build Engineering, Deterministic Checks, CI/CD, Sentry Seer, Sentry

Source: https://www.youtube.com/watch?v=E3KbFLAGD6A