The Dark Arts of Skill Engineering — Paul Bakaus, Renaissance Geek
The talk, 'The Dark Arts of Skill Engineering,' argues that effective AI skill development moves beyond simple prompt engineering and into 'harness engineering.' The speaker details advanced techniques—such as using blind sub-agents, implementing anti-attractors, and utilizing mixture-of-experts architectures—to build robust, deterministic, and highly controllable skills. These techniques focus on exploiting the underlying capabilities of the AI model's execution environment (the 'harness') rather than just the input prompt, resulting in skills that are less susceptible to model drift and more reliable across different deployment environments.
Key takeaways
-
Harness Engineering vs. Prompting
0:06
Skills should be treated as extensions of the entire coding/AI harness, not just packaged prompts. This involves leveraging system capabilities like hooks, scripts, and standard output to enforce behavior, which is more reliable than prose instructions. (6:25)
-
Adversarial Sub-Agents
0:08
To achieve balanced critique or review, use two or more sub-agents that are blind to each other's output. This prevents a single model thread from anchoring on its own work or failing to detect obvious flaws. (7:55, 10:11)
-
Anti-Attractors for Divergence
0:17
To prevent model convergence on common or predictable outputs (the 'median'), implement anti-attractors. Techniques include shaving the next predicted token, generating random seeds (e.g., using celebrities for shaders), or running scripts that introduce unexpected input. (16:59)
-
Passive Guardrails via Hooks
0:20
Use hooks (e.g., pre-tool use hooks) to enforce design or code standards (like design linting) by actively preventing the model from writing non-compliant code, rather than relying on post-hoc corrections. (20:38)
-
Compounding Context and Memory
0:30
Skills can be made aware of prior sessions by saving intermediate results (e.g., critique files) in a dedicated folder. This allows subsequent runs to build context and track progress over time, enabling multi-session refactoring. (30:00)
