Quickstart

Prerequisites

Make sure you have Python 3.10+ and Skill Lab installed.

Your First Evaluation

Let’s evaluate a skill using Skill Lab’s static analysis.

Create a sample skill

Create a directory with a SKILL.md file:

mkdir my-skill
cd my-skill

Create SKILL.md with this content:

SKILL.md

---
name: my-skill
description: A sample skill that demonstrates basic functionality.
---

# My Skill

This skill helps with sample tasks.

## Examples

```python
def hello():
    return "Hello from my-skill!"

Usage

Use this skill when you need to demonstrate basic functionality.

</Step>

<Step title="Run your first evaluation">
From the skill directory, run:

```bash
sklab evaluate

You’ll see a rich-formatted report with:

Quality score (0-100)
Check results by dimension
Passed and failed checks
Suggestions for improvement

The sklab evaluate command defaults to the current directory. You can also specify a path: sklab evaluate ./my-skill

Review the results

The output shows:

Quality Score: Overall score based on weighted checks
Dimension Summary: Results grouped by Structure, Naming, Description, Content
Failed Checks: Detailed messages for any issues found
Suggestions: Recommendations for quality improvements

Common Commands

Now that you’ve run your first evaluation, try these commands:

# Returns exit code 0 if valid, 1 if invalid
sklab validate ./my-skill

Understanding the Output

Here’s what the evaluation report includes:

Quality Score

A weighted 0-100 score based on check results:

90-100: Excellent quality
70-89: Good quality, minor improvements needed
50-69: Fair quality, several issues to address
Below 50: Needs significant improvement

Check Dimensions

Checks are organized into 4 dimensions:

Structure: File existence, frontmatter format, standard fields
Naming: Skill name format, directory matching
Description: Required fields, max length, non-empty
Content: Examples, line budget, reference validation

Severity Levels

Each check has a severity:

Error: Must fix for spec compliance
Warning: Important quality issues
Info: Suggestions for best practices

Advanced: Generate Trigger Tests

This requires Anthropic API key setup and the generate extra.

Generate LLM-powered trigger test cases:

Generate test cases

sklab generate ./my-skill

This creates .skill-lab/tests/triggers.yaml with ~13 test cases across 4 trigger types.

Review the tests

Open .skill-lab/tests/triggers.yaml to see:

Explicit triggers (direct skill name)
Implicit triggers (need description)
Contextual triggers (realistic prompts)
Negative triggers (should NOT activate)

Run the tests (optional)

Requires Claude CLI to be installed.

sklab trigger ./my-skill

Practical Examples

CI/CD Validation
Quality Tracking
Export for Agents
Batch Processing

Use in GitHub Actions or other CI:

.github/workflows/validate-skills.yml

name: Validate Skills
on: [push, pull_request]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.10'
      - run: pip install skill-lab
      - run: sklab validate ./skills

Track quality over time:

# Generate JSON report
sklab evaluate ./my-skill --format json --output report.json

# Extract just the score
sklab evaluate ./my-skill --format json | jq '.quality_score'

Export skills for use with AI agents:

# XML format (recommended for Claude)
sklab prompt ./skill-a ./skill-b > skills.xml

# Markdown format
sklab prompt ./my-skill -f markdown > skill.md

# JSON format
sklab prompt ./my-skill -f json > skill.json

Process multiple skills:

# Validate all skills in a directory
for skill in skills/*/; do
  echo "Validating $skill"
  sklab validate "$skill"
done

# Or use find
find skills -name SKILL.md -execdir sklab validate {} \;

Next Steps

Now that you’ve run your first evaluation, explore more features:

Static Analysis Guide

Deep dive into the 19 static checks

Trigger Testing Guide

Learn about trigger testing in detail

Quality Scoring

Understand how scores are calculated

Output Formats

Master console and JSON output

Skill Format

Learn the SKILL.md format specification

Check Catalog

Browse all available checks

Common Issues

No SKILL.md found

Make sure you’re in a directory with a SKILL.md file, or specify the path:

sklab evaluate ./path/to/skill

Frontmatter parsing errors

Ensure your frontmatter is valid YAML:

---
name: my-skill
description: This is a description.
---

Common issues:

Missing closing ---
Invalid YAML syntax
Non-string field values

Low quality score

Run with --verbose to see all checks:

sklab evaluate ./my-skill --verbose

This shows both passing and failing checks, helping you identify improvements.

Pro tip: Use sklab list-checks --spec-only to see only the checks required by the Agent Skills spec, excluding quality suggestions.

Get Started

Guides

Core Concepts

Development

Prerequisites

Your First Evaluation

Usage

Common Commands

Understanding the Output

Advanced: Generate Trigger Tests

Practical Examples

Next Steps

Static Analysis Guide

Trigger Testing Guide

Quality Scoring

Output Formats

Skill Format

Check Catalog

Common Issues

Build docs developers (and LLMs) love

Get Started

Guides

Core Concepts

Development

​Prerequisites

​Your First Evaluation

​Usage

​Common Commands

​Understanding the Output

​Advanced: Generate Trigger Tests

​Practical Examples

​Next Steps

Static Analysis Guide

Trigger Testing Guide

Quality Scoring

Output Formats

Skill Format

Check Catalog

​Common Issues

Build docs developers (and LLMs) love

Prerequisites

Your First Evaluation

Usage

Common Commands

Understanding the Output

Advanced: Generate Trigger Tests

Practical Examples

Next Steps

Common Issues