The Video Signal

Topic

System Scaling

All digests tagged System Scaling

· 18:09

How Forward Deployed Engineering is done at Decagon — Sunny Rekhi

The talk details the function and evolution of Forward Deployed Engineering (FDE) at Decagon, a company specializing in AI customer service agents. FDE is described as being functionally identical to product engineering, requiring engineers to not only configure complex agent workflows for specific enterprises but also to proactively identify and build platform features that solve anticipated problems across multiple clients. The core philosophy emphasizes architectural restraint: ensuring all custom work is designed to be self-serve and compound into the overall platform, allowing the solution to scale from bespoke deployments to a generalized product.

Key takeaways

  1. FDE Blurs with Product Engineering

    The line between forward deployed service and internal product development is highly blurred. When an enterprise expresses a pain point, it should be treated as a potential product feature that needs prioritization for the entire platform (13:00).

  2. Focus on Architectural Restraint 13:40

    A critical skill in scaling FDE is exercising restraint—avoiding simple one-off patches and instead architecting solutions so they benefit future, unknown customers. This ensures the solution does not become a brittle 'black box' (8:20).

  3. Custom Work Must Compound

    The goal of every deployment is to ensure that custom integrations or workflows are systematically fed back into the platform, transforming bespoke solutions into self-service capabilities for all customers (14:18).

Watch on YouTube Full article

· 21:39

Building Closed-Loop Evals for a Multimodal Agent at Scale — Soumya Gupta & Jai Chopra, Uber

Uber detailed its approach to building closed-loop evaluation systems for a multimodal agent designed to enhance food photography on Uber Eats. The system addresses the challenge of improving image quality at massive scale (90 billion run rate/year) while preserving merchant authenticity and avoiding homogenization. The architecture involves an end-to-end pipeline incorporating Image Understanding/Routing agents, iterative Image Editing agents, multi-dimensional QA gates, and sophisticated continuous learning loops that detect drift and auto-tune the system without human intervention.

Key takeaways

  1. Closed-Loop System Design 17:07

    The system operates on a closed feedback loop: production data is sampled, compared against human labels (golden source of truth), mismatches are diagnosed by an 'Umbrella Diagnosis Agent,' and the auto-tuning pipeline adjusts agent configurations. This process ensures continuous improvement without requiring constant human intervention.

  2. Handling Model Drift 20:03

    To prevent static models from failing in a live environment, the system uses production data sampling to detect drift. The 'Diagnoser' agent localizes the issue and triggers an auto-tuning pipeline that benchmarks against the golden dataset before deployment.

  3. Evaluation Metrics for Agents 21:05

    For routing agents, evaluation involves measuring precision and recall using a confusion matrix (potentially an N x N matrix if multiple branching paths exist). For the enhancement process, 'Pass at K' measures the pass rate after K iterations of refinement.

Watch on YouTube Full article