# Training Taste — Thais Castello Branco, Taste Labs

## Executive summary

Taste Labs addresses the problem of 'AI slop'—the homogenization and lack of context in AI-generated content—by proposing a shift in focus from model training to the application layer (inference time). The core methodology involves quantifying subjective domains like design by training 'probes' (small classifiers) on massive datasets (over 2 million websites). This approach allows for the measurement and prediction of slop, which is superior to traditional LLM-as-a-judge methods. Solutions include the 'Brand API' for structuring brand guidelines into machine-readable components and the 'Creativity API' for intentionally generating out-of-distribution content while respecting domain rules.

## Key takeaways

- Defining Slop and Greatness: Slop is defined by three characteristics: repetition, lack of fit (contextual incoherence), and low intent. While defining 'greatness' is subjective, defining slop is easier, as it represents a general sense of soullessness and convergence.
- Measuring Slop Quantitatively: Taste Labs analyzed over 2 million websites from the past decade to understand design trends. They developed 'probes'—small classifiers—that extract objective features (e.g., contrast, alignment, palette) to predict slop, achieving higher accuracy than LLM-as-a-judge methods.
- Focusing on Inference Time: The most critical intervention point is the application layer (inference time), where user context and intent are exchanged. Improving quality here is considered equally, if not more, important than improving the base model itself.

## Technical details

- Slop Detection Methodology: The team trained 'probes' (small classifiers) on mined features (colors, typography, layout) from large datasets. By combining the frequency of multiple probes, they can quantitatively predict slop, which was shown to outperform LLM-as-a-judge methods.
- Brand API: This product takes a brand URL and extracts its aesthetic rules into structured components. This allows agents to follow brand guidelines while providing a crucial mechanism for human judgment and verification to ensure adherence and quality.
- Creativity API: A system designed to intentionally produce content that is 'out of distribution' rather than merely random. It aims to break rules creatively while maintaining adherence to the general expectations of a specific domain (e.g., a pitch deck).
- Brand Repository: A proposed index of pre-created, cohesive brand systems. This allows average consumers to select a 'dreamy' or 'professional' style rather than relying on a potentially poor generative approach.

## Practical implications

- For developers building AI-powered applications, quality control must be implemented at the application layer (inference time) rather than solely relying on model improvements.
- Implementing structured component extraction (like the Brand API) allows complex, subjective rules (like brand identity) to be enforced programmatically, creating measurable quality gates.
- The concept of 'probes' offers a scalable, data-driven method for evaluating AI output quality across diverse domains.

## Topics

AI Ethics, Generative AI, Design Systems, Machine Learning, Content Quality Control, Taste Labs

Source: https://www.youtube.com/watch?v=sDMGWK4wZ_w