# Why The Prompt Matters Less Than The Context

## Executive summary

The discussion analyzes the effectiveness of various system prompts for Large Language Models (LLMs), concluding that the surrounding context is often more influential than the specific, highly structured wording of the prompt itself. The speaker argues that overly elaborate or 'mannered' prompts (which use phrases like 'parameter worth varying' or 'dial worth turning') are often imprecise and detrimental. Instead, the focus should be on providing clear, literal context and examples to guide the model's output.

## Key takeaways

- Context Over Prompt Wording: The performance of the LLM was found to be highly dependent on the context provided, suggesting that the prompt itself may not be the primary determinant of quality. The speaker notes that the 'unmannered' prompt performed surprisingly well, challenging the assumption that complex, handcrafted rules are always superior.
- Critique of 'Mannered' Prompts: The speaker criticizes overly formal or 'mannered' prompts, arguing that they are often imprecise, use unnecessary jargon, and force the reader to work harder for the writer's performance. The fix suggested is to use literal phrases when available.
- Inconclusive Findings: The analysis of prompt effectiveness was deemed inconclusive due to limited data (e.g., 'only looked at five examples'), indicating that there is no single 'winner' prompt structure.

## Technical details

- Prompt Engineering Comparison: Three prompt types were compared: no system prompt, 'unmannered,' and 'plain writing.' The 'unmannered' prompt showed strong performance, achieving 17 out of 22 points, compared to 14 out of 22 for the 'mannered' version.
- LLM Benchmarking: The discussion references the importance of robust evaluation methods, citing the historical work on CodeSearchNet, a benchmark for semantic code search, which was used by OpenAI to evaluate early code embeddings.

## Practical implications

- When designing prompts, prioritize clear, literal instructions and context over elaborate, highly structured language.
- Treat prompt engineering as a context-setting exercise rather than a rules-writing exercise.
- Be cautious about generalizing prompt findings, as results can be highly dependent on the specific task and dataset size.

## Topics

Prompt Engineering, Large Language Models (LLMs), System Prompts, AI Evaluation (Evals), Natural Language Processing (NLP), AI Evals October 2026 cohort

Source: https://www.youtube.com/watch?v=CoEqTMTqFSQ