How I automate my own job at Hugging Face using agents — Niels Rogge, Hugging Face
Summary
Niels Rogge details how he automated his role at Hugging Face—the 'Google Drive to the hub' team—which focuses on improving the discoverability of machine learning artifacts. He built two systems: an initial deterministic workflow for outreach (using cron jobs and LLM APIs) and a subsequent fully autonomous agent loop for follow-up actions. The architecture leverages modern tooling like Modal, Bash CLI skills, and advanced models (e.g., GLM 5.2) to scale the process of identifying missing artifacts and prompting researchers to publish them on Hugging Face.
Key takeaways
-
The Problem: Artifact Discoverability
ML weights and datasets are often published on third-party services (Google Drive, Zenodo) rather than the centralized platform (Hugging Face), hindering discoverability. The goal is to automate outreach to authors.
-
Initial Automation: Deterministic Workflow
11:43
The first phase used a deterministic workflow, running as a nightly cron job on GitHub Actions. This approach utilized LLM APIs in predefined steps without an agent framework, offering high predictability and control.
-
Advanced Automation: Autonomous Agent Loop
15:36
The follow-up process was automated using a fully autonomous agent loop (e.g., leveraging the Claude agents SDK). This flexible approach allows the agent to use tools and skills, such as Bash and the Hugging Face CLI, to interact with GitHub issues.
Technical details
-
Workflow Architecture
703s
The initial outreach workflow was implemented using a Python script running as a cron job on GitHub Actions, calling LLM APIs without an agent framework to ensure determinism. LangFuse is used for observability, tracing inputs, outputs, prompts, cost, and latency.
-
Agent Architecture
1025s
The follow-up system uses a fully autonomous agent loop (e.g., Claude agents SDK). It is deployed on Modal using batch processing to spin up massive amounts of containers, with each container handling one GitHub issue. Tools include Bash and the Hugging Face CLI skill.
-
Model/Tooling Stack
The system utilizes open models like GLM 5.2 via Hugging Face inference providers, which unifies access to various LLMs (OpenAI compatible or Anthropic compatible). The process relies on the Hugging Face CLI skill for artifact management and GitHub interaction.
Mentioned resources
- LangFuse
- GitHub Actions
- Modal
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.