Topic

Skill Automation

All digests tagged Skill Automation

We Scored a Real Snyk Skill Against Anthropic's Rules thumbnail

· 15:19

We Scored a Real Snyk Skill Against Anthropic's Rules

This video details a live review process where a Snyk skill (`SKILL.md`) was evaluated using Tessl's `tessl review run` against Anthropic's best practices. The initial score of 87% was successfully improved to 90% by applying fixes, demonstrating how automated tools can enhance skill quality and security. Key focus areas include implementing progressive disclosure to prevent context bloat, improving skill conciseness, and using Snyk's Agent Scan tool to detect vulnerabilities like prompt injection in both first-party and third-party skills.

Key takeaways

  1. Skill Quality Improvement via Automated Review

    The review process successfully increased the skill score from 87% to 90% by applying fixes, demonstrating that automated tools can significantly improve adherence to best practices (e.g., Anthropic's guidelines).

  2. Importance of Progressive Disclosure 10:08

    Skills should not be overly dense or verbose. Implementing progressive disclosure—breaking large skills into smaller, referenced sub-files—prevents context bloat and ensures the agent only loads necessary information.

  3. Security Scanning with Agent Scan

    Snyk's dedicated tool, Agent Scan (available on GitHub), can scan skills for security vulnerabilities, including prompt injection, which is crucial when integrating third-party or user-written skills.

Watch on YouTube Full article

We Scored Oracle's Database Skill Live: 95% Isn't Enough thumbnail

· 14:35

We Scored Oracle's Database Skill Live: 95% Isn't Enough

The video demonstrates how Oracle's database team utilized AI 'skills'—encapsulated in a repository like `oracle/skills`—to guide AI agents in writing secure and best-practice compliant SQL. The process involved running an automated review (using Tessl) on a skill designed to improve database code generation, which initially scored 95%. Through iterative refinement and applying changes via a 'run, review, fix' cycle, the skill was successfully upgraded to a 100% success rate, showcasing a robust mechanism for encoding complex domain knowledge into AI workflows.

Key takeaways

  1. Purpose of Database Skills 5:51

    The primary function of the skill is to help agents write better SQL and database code by making them knowledgeable about the database's capabilities (e.g., property graphs, vector data). It helps discover what the database can do, rather than relying solely on general agent knowledge.

  2. Skill Review Process 13:44

    The skill was subjected to an automated review comparing it against best practices. The process involves running a 'run, review, fix' cycle, which applies changes and re-runs the validation in an agile style approach.

  3. Achieving 100% Compliance 14:35

    After making specific developer-suggested changes (e.g., adding a recommended sequence task), the skill's review score reached 100%, demonstrating continuous improvement and validation of domain knowledge.

Watch on YouTube Full article