The Video Signal

Topic

Machine Learning Pipelines

All digests tagged Machine Learning Pipelines

Skill issue: stop deploying vision language models, use them with Skills — Merve Noyan, Hugging Face thumbnail

· 19:14

Skill issue: stop deploying vision language models, use them with Skills — Merve Noyan, Hugging Face

The talk argues that developers should stop using Vision Language Models (VLMs) directly for real-time computer vision tasks, as they are often insufficient for high-performance, low-latency applications. Instead, the speaker introduces a comprehensive toolkit and methodology for building robust, task-specific detectors (like RFDTR) using Apache 2.0 licensed models. A key innovation is the 'vibe training' pipeline, which leverages VLMs as labelers and judges to efficiently train specialized models when only unlabeled images are available. The entire process is designed to be scalable and cost-effective, running on cloud infrastructure.

Key takeaways

  1. VLM Limitations for Real-Time CV

    VLMs are unsuitable for real-time applications; specialized detectors (e.g., RFDTR) running on hardware like a toaster can achieve 30-40 FPS, outperforming VLMs.

  2. Licensing Compliance 2:31

    Developers must prioritize using Apache 2.0 licensed models and avoid deploying popular detectors (like YOLO) without checking their copyleft licenses (e.g., AGPL 3.0).

  3. Vibe Training Pipeline 5:41

    A novel pipeline uses a VLM as a labeler and a second VLM as a judge to annotate and evaluate data, enabling the training of specialized detectors (RFDTR) from unlabeled datasets.

Watch on YouTube Full article

Training Agents 3: Reinforcement Learning thumbnail

· 1:17:20

Training Agents 3: Reinforcement Learning

This session introduces Group Relative Policy Optimization (GRPO), a reinforcement learning (RL) method that advances agent training beyond mere imitation (SFT/Distillation). GRPO trains models by sampling multiple completions per prompt and using the group's relative scores—calculated via a reward function—as the primary training signal. This approach is highly effective for complex tasks, allowing agents to learn from their own varied trajectories in an iterative loop.

Key takeaways

  1. GRPO Mechanics 20:30

    GRPO calculates advantages relative to the group average (Reward - Group Average / Group Standard Deviation). This method eliminates the need for a separate value model, reward model, or critic, simplifying the RL loop. The process involves generating multiple rollouts, scoring them with a verifiable Python function (the reward function), and updating the policy based on these relative advantages.

  2. Training Pipeline Progression 5:05

    The training pipeline typically progresses from Supervised Fine-Tuning (SFT) for dense, off-policy signals, to Distillation for richer online rollouts, and finally to RL/GRPO for sparse, on-policy learning. SFT is often used first to bootstrap the model's understanding of the task structure.

  3. Reward Function Design 26:40

    Defining a verifiable reward function is critical; it acts as a 'contract' defining success. Rewards can be composed of multiple components, such as a format check (e.g., ensuring JSON structure) and an accuracy check (e.g., passing unit tests). For agentic tasks, using test suites or compiling code provides robust signals.

  4. Interpreting Training Curves 35:00

    Monitoring training curves via tools like Track.io is essential for debugging. Key metrics include the reward (should rise), entropy (should remain stable/flat, not dive or spike), and completion length. Failure modes—such as 'reward hacking' (high reward but low test accuracy) or 'collapse' (low entropy)—require deep data inspection.

Watch on YouTube Full article