Skip to main content
Skill Lab organizes its 28 static checks into 4 primary dimensions that categorize different aspects of skill quality. Each dimension focuses on a specific concern and contributes a weighted percentage to the overall quality score.

​
Overview

From core/models.py:17-24: 
class EvalDimension(str, Enum):
    """Evaluation dimensions for categorizing checks."""
    
    STRUCTURE = "structure"
    NAMING = "naming"
    DESCRIPTION = "description"
    CONTENT = "content"
    EXECUTION = "execution"  # Phase 3: Trace-based checks

​
1. Structure (30% weight)

Focus: File organization, folder validity, and frontmatter structure. The Structure dimension ensures that the skill follows the correct file and folder layout, contains valid YAML frontmatter, and uses appropriate file types.

​
Checks in This Dimension

Check ID: structure.skill-md-exists
Severity: ERROR
Spec Required: YesVerifies that a SKILL.md file exists in the skill directory. Also warns if skill.md (lowercase) is found instead.Source: structure.py:18-48
Check ID: structure.valid-frontmatter
Severity: ERROR
Spec Required: YesValidates that the YAML frontmatter can be parsed without errors and is not empty.Source: structure.py:52-82
Check ID: structure.standard-frontmatter-fields
Severity: WARNING
Spec Required: NoChecks that frontmatter only contains spec-defined fields. Custom fields should be placed in the metadata map.Spec Fields: name, description, license, compatibility, metadata, allowed-toolsSource: structure.py:177-227
Check ID: structure.scripts-valid
Severity: WARNING
Spec Required: NoEnsures /scripts contains only valid script files with extensions: .py, .sh, .js, .ts, .bash, .rbSource: structure.py:86-127
Check ID: structure.references-valid
Severity: WARNING
Spec Required: NoEnsures /references contains only valid reference files with extensions: .md, .txt, .rstSource: structure.py:131-172
Check ID: structure.scripts-no-interactive
Severity: WARNING
Spec Required: NoDetects interactive input patterns that won’t work in non-interactive shells:
  • Python: input(), getpass.getpass()
  • Shell: read, select
  • Ruby: gets, STDIN.gets
  • JS/TS: readline, prompt(), process.stdin
Source: structure.py:257-311
Check ID: structure.scripts-self-contained
Severity: INFO
Spec Required: NoWarns if /scripts contains dependency manifests like requirements.txt, package.json, Gemfile, etc.Scripts should use inline metadata (PEP 723) or install dependencies within the script.Source: structure.py:324-359

​
2. Naming (20% weight)

Focus: Skill name format and consistency. The Naming dimension ensures that skill names follow the spec’s kebab-case convention and match the parent directory name for consistency.

​
Checks in This Dimension

Check ID: naming.matches-directory
Severity: ERROR
Spec Required: YesValidates that the name field in frontmatter matches the parent directory name (after Unicode normalization).Source: naming.py:12-42

​
Schema-Based Naming Checks

Additional naming constraints are enforced through schema-based checks (see schema.py:76-112):
  • Name Required (naming.required) - Name field must be present
  • Name Format (naming.format) - Must be lowercase, hyphen-separated, max 64 chars, no consecutive hyphens

​
3. Description (25% weight)

Focus: Description field presence, length, and quality. The Description dimension ensures that skills have clear, concise descriptions that help agents understand when to use them.

​
Checks in This Dimension

All description checks are schema-based (from schema.py:114-155):
Check ID: description.required
Severity: ERROR
Spec Required: YesEnsures the description field exists in frontmatter.
Check ID: description.not-empty
Severity: ERROR
Spec Required: YesValidates that description is not empty or whitespace-only.
Check ID: description.max-length
Severity: ERROR
Spec Required: YesEnsures description is under 1024 characters.

​
Content-Based Description Check

One additional check in the Content dimension analyzes description quality:
  • Description Actionable (content.description-actionable) - Checks for activation phrases like “Use when…”, “Designed for…“

​
4. Content (25% weight)

Focus: SKILL.md body quality, examples, token budgets, and resource references. The Content dimension evaluates the actual instructions and documentation in the skill body.

​
Checks in This Dimension

Check ID: content.body-not-empty
Severity: WARNING
Spec Required: NoEnsures the SKILL.md body has meaningful content (at least 50 characters).Source: content.py:50-79
Check ID: content.line-budget
Severity: WARNING
Spec Required: NoValidates that the body is under 500 lines.Source: content.py:83-106
Check ID: content.has-examples
Severity: INFO
Spec Required: NoChecks for code examples using patterns: fenced code blocks (```), indented code blocks, or <example> tags.Source: content.py:110-133
Check ID: content.reference-depth
Severity: WARNING
Spec Required: NoEnsures references are maximum 1 level deep to avoid excessive nesting.Source: content.py:137-178
Check ID: content.scripts-referenced
Severity: WARNING
Spec Required: NoVerifies that scripts in /scripts are mentioned in the body so the agent knows they exist.Source: content.py:182-225
Check ID: content.script-paths-exist
Severity: WARNING
Spec Required: NoValidates that script paths referenced in the body (e.g., scripts/setup.py) exist on disk.Source: content.py:234-272
Check ID: content.compatibility-prereqs
Severity: INFO
Spec Required: NoChecks that command runners in the body (like npx, uvx, bunx) are documented in the compatibility field.Source: content.py:293-346
Check ID: content.token-budget
Severity: WARNING
Spec Required: NoEnsures the body stays under the spec-recommended 5,000 token budget.Source: content.py:350-372
Check ID: content.metadata-token-budget
Severity: INFO
Spec Required: NoValidates that combined name + description is under 150 tokens for efficient discovery.Source: content.py:376-407
Check ID: content.description-actionable
Severity: INFO
Spec Required: NoChecks that the description includes activation phrasing (“Use when…”, “Designed for…”, etc.).Activation Phrases: use when, use for, use this, trigger, activate, invoke, run when, run this, helps with, designed for, intended for, works withSource: content.py:411-442
Check ID: content.asset-paths-exist
Severity: WARNING
Spec Required: NoValidates that asset paths referenced in the body (e.g., assets/template.json) exist on disk.Source: content.py:450-488

​
5. Execution (0% weight in static analysis)

Focus: Runtime behavior from trace analysis (Phase 3). The Execution dimension is reserved for trace-based checks that analyze how skills perform during actual execution. These checks are run separately using sklab trace and do not contribute to the static analysis quality score.
Execution dimension checks are part of Phase 3 (Trace Analysis) and are not yet fully implemented in v0.4.0.

​
Listing Checks by Dimension

To see all checks for a specific dimension: 
sklab list-checks --dimension structure
sklab list-checks --dimension naming
sklab list-checks --dimension description
sklab list-checks --dimension content
To see all checks: 
sklab list-checks

​
Dimension Score Breakdown

When running an evaluation, the summary includes dimension-level scores: 
sklab evaluate ./my-skill
Example output: 
Dimension Scores:
  structure:    85.0% (6/7 passed)
  naming:      100.0% (1/1 passed)
  description: 100.0% (3/3 passed)
  content:      80.0% (8/11 passed)

Quality Score: 88.75/100
See Quality Scoring for details on how dimension scores are calculated and weighted.

Build docs developers (and LLMs) love

Get started for freeTalk to us