name: better-skill-creator description: Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, optimize a skill's description for better triggering accuracy, or package and distribute a finished skill. Also use when users say things like "turn this into a skill", "make a skill for X", "help me build a workflow", "improve my skill", or "test this skill". Covers all three skill categories — document/asset creation skills, workflow automation skills, and MCP-enhanced skills.

Skill Creator

A skill for creating new skills and iteratively improving them.

Overview

At a high level, the process of creating a skill goes like this:

Decide what the skill should do and roughly how it should do it
Classify which category it falls into (this shapes the structure)
Write a draft of the skill
Create a few test prompts and run claude-with-access-to-the-skill on them
Help the user evaluate the results both qualitatively and quantitatively
- While the runs happen in the background, draft some quantitative evals if there aren't any (if there are some, you can either use as is or modify). Then explain them to the user
- Use the eval-viewer/generate_review.py script to show the user the results for review, and also let them look at the quantitative metrics
Rewrite the skill based on feedback from the user's evaluation (and any glaring flaws from quantitative benchmarks)
Repeat until satisfied
Expand the test set and try again at larger scale
Run the pre-flight checklist to validate everything
Package and deliver

Your job is to figure out where the user is in this process and help them progress. Maybe they want to make a skill from scratch. Maybe they already have a draft. Maybe they have a working skill that undertriggers. Be flexible and meet them where they are.

If the user says "I don't need to run a bunch of evaluations, just vibe with me", you can absolutely do that instead.

Communicating with the user

The skill creator is used by people across a wide range of technical familiarity. Pay attention to context cues about how to phrase things. In the default case:

"evaluation" and "benchmark" are borderline, but OK
For "JSON" and "assertion", see cues from the user that they know what those things are before using them without explaining

It's OK to briefly explain terms if you're in doubt. If the user seems non-technical, focus on outcomes rather than implementation details.

Phase 1: Planning and Design

Capture Intent

Start by understanding the user's intent. The current conversation might already contain a workflow the user wants to capture (e.g., they say "turn this into a skill"). If so, extract answers from the conversation history first — the tools used, the sequence of steps, corrections the user made, input/output formats observed. The user may need to fill gaps, and should confirm before proceeding.

Key questions to answer:

What should this skill enable Claude to do?
When should this skill trigger? (what user phrases/contexts)
What's the expected output format?
Should we set up test cases? Skills with objectively verifiable outputs (file transforms, data extraction, code generation, fixed workflow steps) benefit from test cases. Skills with subjective outputs (writing style, art) often don't need them. Suggest the appropriate default based on the skill type, but let the user decide.

Classify the Skill Category

Every skill falls into one (or a blend) of three categories. Identifying the category early shapes the structure, patterns, and testing approach. See references/categories-and-patterns.md for full details, but briefly:

Category 1: Document & Asset Creation — Creating consistent, high-quality outputs (documents, presentations, apps, designs, code). No external tools required, uses Claude's built-in capabilities. Key techniques: embedded style guides, template structures, quality checklists.

Category 2: Workflow Automation — Multi-step processes that benefit from consistent methodology, potentially coordinating across multiple MCP servers. Key techniques: step-by-step workflows with validation gates, iterative refinement loops, built-in review suggestions.

Category 3: MCP Enhancement — Workflow guidance that enhances the tool access an MCP server provides. Key techniques: coordinates multiple MCP calls in sequence, embeds domain expertise, provides context users would otherwise need to specify.

Ask the user which category fits best. Many skills blend categories — that's fine. The category just shapes which patterns to emphasize.

Define Use Cases

Before writing any code, have the user identify 2-3 concrete use cases. Each use case should follow this structure:

Use Case: [Name]
Trigger: [What the user says or does]
Steps:
1. [First step]
2. [Second step]
...
Result: [What success looks like]

Define Success Criteria

Help the user think about how they'll know the skill is working. These are aspirational targets — rough benchmarks, not precise thresholds.

Quantitative (when applicable):

Skill triggers on 90%+ of relevant queries
Completes workflow in X tool calls (compare with/without skill)
0 failed API calls per workflow

Qualitative:

Users don't need to prompt Claude about next steps
Workflows complete without user correction
Consistent results across sessions

Interview and Research

Proactively ask questions about edge cases, input/output formats, example files, success criteria, and dependencies. Wait to write test prompts until this is ironed out.

Check available MCPs — if useful for research (searching docs, finding similar skills, looking up best practices), research in parallel via subagents if available, otherwise inline. Come prepared with context to reduce burden on the user.

Phase 2: Writing the Skill

Skill Architecture

skill-name/
├── SKILL.md          # Required — main skill file
├── scripts/          # Optional — executable code
│   ├── process_data.py
│   └── validate.sh
├── references/       # Optional — documentation loaded as needed
│   ├── api-guide.md
│   └── examples/
└── assets/           # Optional — templates, fonts, icons
    └── report-template.md

Progressive Disclosure (Three Levels)

Skills use a three-level loading system — this is critical for token efficiency:

Metadata (name + description) — Always in context (~100 words). This is how Claude decides whether to load the skill.
SKILL.md body — In context whenever skill triggers (<500 lines ideal). Contains the full instructions.
Bundled resources — Loaded as needed (unlimited size, scripts execute without loading into context).

Key constraints:

Keep SKILL.md under 500 lines; if approaching this limit, add hierarchy with clear pointers to referenced files
Reference files clearly from SKILL.md with guidance on when to read them
For large reference files (>300 lines), include a table of contents

Critical Rules

SKILL.md naming:

Must be exactly SKILL.md (case-sensitive)
No variations accepted (SKILL.MD, skill.md, Skill.md, etc.)

Skill folder naming:

Use kebab-case: notion-project-setup ✅
No spaces: Notion Project Setup ❌
No underscores: notion_project_setup ❌
No capitals: NotionProjectSetup ❌

No README.md inside the skill folder. All documentation goes in SKILL.md or references/. (A repo-level README for GitHub distribution is separate.)

Security restrictions — forbidden in frontmatter:

XML angle brackets (< >) — because frontmatter appears in Claude's system prompt, these could inject instructions
Skills named with "claude" or "anthropic" prefix (reserved)

Writing the YAML Frontmatter

The frontmatter is the most important part of a skill — it's how Claude decides whether to load it. Get this right.

Required fields:

---
name: your-skill-name
description: What it does. Use when user asks to [specific phrases].
---

Optional fields:

license: e.g., MIT, Apache-2.0
compatibility: Environment requirements (1-500 chars)
allowed-tools: Restrict tool access, e.g., "Bash(python:*) Bash(npm:*) WebFetch"
metadata: Custom key-value pairs (author, version, mcp-server, category, tags)

Writing the description field:

Structure: [What it does] + [When to use it] + [Key capabilities]

The description is the primary triggering mechanism. Claude has a tendency to undertrigger — to not use skills when they'd be useful. To combat this, make descriptions slightly "pushy" and include diverse trigger phrases.

Good descriptions:

# Specific and actionable
description: Analyzes Figma design files and generates developer handoff documentation. Use when user uploads .fig files, asks for "design specs", "component documentation", or "design-to-code handoff".

# Includes trigger phrases and scope
description: Manages Linear project workflows including sprint planning, task creation, and status tracking. Use when user mentions "sprint", "Linear tasks", "project planning", or asks to "create tickets".

# Clear value proposition with negative triggers
description: Advanced data analysis for CSV files. Use for statistical modeling, regression, clustering. Do NOT use for simple data exploration (use data-viz skill instead).

Bad descriptions:

# Too vague — won't trigger on anything specific
description: Helps with projects.

# Missing triggers — Claude doesn't know *when* to use it
description: Creates sophisticated multi-page documentation systems.

# Too technical, no user triggers
description: Implements the Project entity model with hierarchical relationships.

Description character limit: 1024 characters. Make every character count.

Writing the Instructions Body

After the frontmatter, write the actual instructions. Consult references/categories-and-patterns.md to pick the right pattern for the skill category, then adapt this template:

---
name: your-skill
description: [...]
---

# Your Skill Name

## Instructions

## Step 1: [First Major Step]
Clear explanation of what happens.

Example:
\`\`\`bash
python scripts/fetch_data.py --project-id PROJECT_ID
\`\`\`
Expected output: [describe what success looks like]

(Add more steps as needed)

## Examples
Example 1: [common scenario]
User says: "Set up a new marketing campaign"
Actions:
1. Fetch existing campaigns via MCP
2. Create new campaign with provided parameters
Result: Campaign created with confirmation link

## Troubleshooting
Error: [Common error message]
Cause: [Why it happens]
Solution: [How to fix]

Writing Style Principles

Use the imperative form in instructions. Say "Run the validation script" not "You should run the validation script."
Explain the why. Today's LLMs are smart. They have good theory of mind and when given a good harness can go beyond rote instructions. If you find yourself writing ALWAYS or NEVER in all caps, or using super rigid structures, that's a yellow flag — reframe and explain the reasoning so the model understands why the thing is important.
Be specific and actionable.
- ✅ Good: Run python scripts/validate.py --input {filename} to check data format. If validation fails, common issues include: missing required fields, invalid date formats (use YYYY-MM-DD).
- ❌ Bad: Validate the data before proceeding.
Include error handling — anticipate what goes wrong and tell the model what to do about it.
Use progressive disclosure — keep SKILL.md focused on core instructions. Move detailed documentation to references/ and link to it.
Look for repeated work across test cases. If all test runs independently write similar helper scripts, that's a signal the skill should bundle that script in scripts/.
Keep the prompt lean. Remove things that aren't pulling their weight. Read transcripts, not just final outputs — if the skill makes the model waste time on unproductive steps, trim those instructions.

Domain Organization

When a skill supports multiple domains/frameworks, organize by variant:

cloud-deploy/
├── SKILL.md (workflow + selection)
└── references/
    ├── aws.md
    ├── gcp.md
    └── azure.md

Claude reads only the relevant reference file.

Principle of Lack of Surprise

Skills must not contain malware, exploit code, or any content that could compromise system security. A skill's contents should not surprise the user in their intent. Don't go along with requests to create misleading skills or skills designed to facilitate unauthorized access, data exfiltration, or other malicious activities.

Phase 3: Testing and Iteration

Test Cases

After writing the skill draft, come up with 2-3 realistic test prompts — the kind of thing a real user would actually say. Share them with the user: "Here are a few test cases I'd like to try. Do these look right, or do you want to add more?" Then run them.

Save test cases to evals/evals.json. Don't write assertions yet — just the prompts. You'll draft assertions in the next step while runs are in progress.

{
  "skill_name": "example-skill",
  "evals": [
    {
      "id": 1,
      "prompt": "User's task prompt",
      "expected_output": "Description of expected result",
      "files": []
    }
  ]
}

See references/schemas.md for the full schema (including the assertions field).

Three Types of Tests

1. Triggering tests — Does the skill load at the right times?

✅ Triggers on obvious tasks
✅ Triggers on paraphrased requests
❌ Doesn't trigger on unrelated topics

2. Functional tests — Does the skill produce correct outputs?

Valid outputs generated
API calls succeed
Error handling works
Edge cases covered

3. Performance comparison — Does the skill improve results vs. baseline?

Compare tool calls, tokens, time with/without skill
Track pass rates across iterations

Running and Evaluating Test Cases

This section is one continuous sequence — don't stop partway through. Do NOT use /skill-test or any other testing skill.

Put results in <skill-name>-workspace/ as a sibling to the skill directory. Within the workspace, organize results by iteration (iteration-1/, iteration-2/, etc.) and within that, each test case gets a directory (eval-0/, eval-1/, etc.). Create directories as you go.

Step 1: Spawn all runs (with-skill AND baseline) in the same turn

For each test case, spawn two subagents in the same turn — one with the skill, one without. Launch everything at once so it all finishes around the same time.

With-skill run:

Execute this task:
- Skill path: <path-to-skill>
- Task: <eval prompt>
- Input files: <eval files if any, or "none">
- Save outputs to: <workspace>/iteration-<N>/eval-<ID>/with_skill/outputs/
- Outputs to save: <what the user cares about>

Baseline run (same prompt, but the baseline depends on context):

Creating a new skill: no skill at all. Same prompt, no skill path, save to without_skill/outputs/.
Improving an existing skill: the old version. Before editing, snapshot the skill (cp -r <skill-path> <workspace>/skill-snapshot/), then point baseline at the snapshot. Save to old_skill/outputs/.

Write an eval_metadata.json for each test case. Give each eval a descriptive name based on what it's testing. If this iteration uses new or modified eval prompts, create these files for each new eval directory.

{
  "eval_id": 0,
  "eval_name": "descriptive-name-here",
  "prompt": "The user's task prompt",
  "assertions": []
}

Step 2: While runs are in progress, draft assertions

Don't just wait — use this time productively. Draft quantitative assertions for each test case and explain them to the user. Good assertions are objectively verifiable and have descriptive names that read clearly in the benchmark viewer.

Subjective skills (writing style, design quality) are better evaluated qualitatively — don't force assertions onto things that need human judgment.

Update the eval_metadata.json files and evals/evals.json with the assertions once drafted.

Step 3: As runs complete, capture timing data

When each subagent task completes, you receive a notification containing total_tokens and duration_ms. Save this data immediately to timing.json in the run directory:

{
  "total_tokens": 84852,
  "duration_ms": 23332,
  "total_duration_seconds": 23.3
}

This is the only opportunity to capture this data — process each notification as it arrives.

Step 4: Grade, aggregate, and launch the viewer

Once all runs are done:

Grade each run — spawn a grader subagent (or grade inline) that reads agents/grader.md and evaluates each assertion. Save results to grading.json. The grading.json expectations array must use text, passed, and evidence fields (not name/met/details). For assertions that can be checked programmatically, write and run a script.
Aggregate into benchmark:
```
python -m scripts.aggregate_benchmark <workspace>/iteration-N --skill-name <n>
```
Put each with_skill version before its baseline counterpart.
Do an analyst pass — read the benchmark data and surface patterns the aggregate stats might hide. See agents/analyzer.md for what to look for.

Launch the viewer:

nohup python <skill-creator-path>/eval-viewer/generate_review.py \
  <workspace>/iteration-N \
  --skill-name "my-skill" \
  --benchmark <workspace>/iteration-N/benchmark.json \
  > /dev/null 2>&1 &
VIEWER_PID=$!

For iteration 2+, also pass --previous-workspace <workspace>/iteration-<N-1>.

Cowork / headless environments: Use --static <output_path> to write a standalone HTML file instead of starting a server.

Note: use generate_review.py to create the viewer; don't write custom HTML.

Tell the user something like: "I've opened the results in your browser. There are two tabs — 'Outputs' for reviewing each test case with feedback, 'Benchmark' for quantitative comparison. When you're done, come back and let me know."

Step 5: Read the feedback

When the user tells you they're done, read feedback.json:

{
  "reviews": [
    {"run_id": "eval-0-with_skill", "feedback": "the chart is missing axis labels", "timestamp": "..."},
    {"run_id": "eval-1-with_skill", "feedback": "", "timestamp": "..."}
  ],
  "status": "complete"
}

Empty feedback means the user thought it was fine. Focus improvements on test cases with specific complaints.

Kill the viewer server when done: kill $VIEWER_PID 2>/dev/null

Phase 4: Improving the Skill

This is the heart of the loop. You've run test cases, the user has reviewed results, and now you need to improve the skill based on their feedback.

How to Think About Improvements

Generalize from the feedback. We're trying to create skills used a million times across many different prompts. You're iterating on only a few examples because it's faster. But if the skill only works for those examples, it's useless. Rather than fiddly overfitty changes or oppressively constrictive MUSTs, if there's a stubborn issue, try different metaphors or different patterns.
Keep the prompt lean. Remove things not pulling their weight. Read the transcripts — if the skill makes the model waste time on unproductive steps, trim those instructions.
Explain the why. Transmit understanding into instructions. If you find yourself writing ALWAYS or NEVER in all caps, reframe and explain the reasoning. That's more humane, powerful, and effective.
Look for repeated work across test cases. If all test runs independently write similar helper scripts, bundle that script in scripts/. This saves every future invocation from reinventing the wheel.
Apply troubleshooting patterns. Consult references/troubleshooting.md to diagnose common issues (undertriggering, overtriggering, instructions not followed, large context degradation).

The Iteration Loop

After improving the skill:

Apply improvements to the skill
Rerun all test cases into a new iteration-<N+1>/ directory, including baseline runs
Launch the reviewer with --previous-workspace pointing at the previous iteration
Wait for the user to review and tell you they're done
Read the new feedback, improve again, repeat

Keep going until:

The user says they're happy
The feedback is all empty (everything looks good)
You're not making meaningful progress

Phase 5: Validation and Polish

Pre-Flight Checklist

Before finalizing, run through this validation checklist. You can also run the automated validator:

python -m scripts.quick_validate <path-to-skill-folder>

Structure checks:

[ ] Folder named in kebab-case
[ ] SKILL.md file exists (exact spelling, case-sensitive)
[ ] YAML frontmatter has --- delimiters
[ ] name field: kebab-case, no spaces, no capitals, no "claude" or "anthropic" prefix
[ ] description includes WHAT it does and WHEN to use it
[ ] No XML tags (< >) anywhere in frontmatter
[ ] No README.md inside skill folder
[ ] Description under 1024 characters

Content checks:

[ ] Instructions are clear and actionable (imperative form)
[ ] Error handling included
[ ] Examples provided for common scenarios
[ ] References clearly linked with guidance on when to read them
[ ] SKILL.md under 500 lines (references handle the rest)

Testing checks:

[ ] Triggers on obvious tasks
[ ] Triggers on paraphrased requests
[ ] Doesn't trigger on unrelated topics
[ ] Functional tests pass
[ ] Tool integration works (if applicable)

Advanced: Blind Comparison

For situations where you want a more rigorous comparison between two versions (e.g., "is the new version actually better?"), there's a blind comparison system. Read agents/comparator.md and agents/analyzer.md for details. This is optional and most users won't need it.

Phase 6: Description Optimization

The description field is the primary mechanism determining whether Claude invokes a skill. After creating or improving a skill, offer to optimize the description for better triggering accuracy.

Step 1: Generate trigger eval queries

Create 20 eval queries — a mix of should-trigger and should-not-trigger. Save as JSON:

[
  {"query": "the user prompt", "should_trigger": true},
  {"query": "another prompt", "should_trigger": false}
]

The queries must be realistic — concrete, specific, with detail. File paths, personal context, column names, company names, URLs, a little backstory. Mix of lengths. Focus on edge cases rather than clear-cut tests.

Bad: "Format this data", "Extract text from PDF", "Create a chart"

Good: "ok so my boss just sent me this xlsx file (its in my downloads, called something like 'Q4 sales final FINAL v2.xlsx') and she wants me to add a column that shows the profit margin as a percentage. The revenue is in column C and costs are in column D i think"

For should-trigger queries (8-10): different phrasings of the same intent, some formal, some casual. Include cases where the user doesn't explicitly name the skill but clearly needs it.

For should-not-trigger queries (8-10): near-misses that share keywords but need something different. Don't make these obviously irrelevant — "Write a fibonacci function" as a negative test for a PDF skill tests nothing. The negatives should be genuinely tricky.

Step 2: Review with user

Present the eval set using the HTML template:

Read assets/eval_review.html
Replace placeholders:
- __EVAL_DATA_PLACEHOLDER__ → the JSON array
- __SKILL_NAME_PLACEHOLDER__ → skill name
- __SKILL_DESCRIPTION_PLACEHOLDER__ → current description
Write to a temp file and open it
User edits queries, toggles should-trigger, then clicks "Export Eval Set"
Check Downloads for the exported file

Step 3: Run the optimization loop

Tell the user: "This will take some time — I'll run the optimization loop in the background."

python -m scripts.run_loop \
  --eval-set <path-to-trigger-eval.json> \
  --skill-path <path-to-skill> \
  --model <model-id-powering-this-session> \
  --max-iterations 5 \
  --verbose

Use the model ID from your system prompt so the triggering test matches what the user experiences.

While it runs, periodically tail output to give updates. This handles the full optimization loop automatically: splits 60% train / 40% test, evaluates current description (3x per query), proposes improvements, re-evaluates, iterates up to 5 times, selects by test score to avoid overfitting.

How skill triggering works

Skills appear in Claude's available_skills list with their name + description, and Claude decides whether to consult based on that description. Claude only consults skills for tasks it can't easily handle on its own — simple, one-step queries like "read this PDF" may not trigger even if the description matches, because Claude handles them directly. Complex, multi-step, or specialized queries reliably trigger skills when the description matches.

Eval queries should be substantive enough that Claude would benefit from consulting a skill.

Step 4: Apply the result

Take best_description from JSON output and update the SKILL.md frontmatter. Show user before/after and report scores.

Phase 7: Package and Distribute

Package and Present (only if `present_files` tool is available)

Check whether you have access to the present_files tool. If you don't, skip this step. If you do:

python -m scripts.package_skill <path/to/skill-folder>

After packaging, direct the user to the resulting .skill file path so they can install it.

Distribution Guidance

For users who want to share their skill:

Host on GitHub — public repo, clear README (separate from skill folder), example usage with screenshots
Document with MCP repo (if MCP skill) — link from MCP docs, explain value of using both together
Create installation guide — step-by-step instructions for Claude.ai upload or Claude Code placement

Remind users to focus on outcomes in their positioning: "enables teams to set up complete project workspaces in seconds" beats "a folder containing YAML frontmatter and Markdown instructions."

Environment-Specific Instructions

Claude.ai

In Claude.ai, the core workflow is the same (draft → test → review → improve → repeat), but no subagents means some mechanics change:

Running test cases: No parallel execution. For each test case, read the skill's SKILL.md, then follow its instructions to accomplish the test prompt yourself. Do them one at a time. Skip baseline runs.

Reviewing results: If no browser is available, present results directly in conversation. For each test case, show the prompt and output. If the output is a file, save to the filesystem and tell the user where it is. Ask for feedback inline.

Benchmarking: Skip quantitative benchmarking — it relies on baselines which aren't meaningful without subagents. Focus on qualitative feedback.

Description optimization: Requires claude -p (CLI tool), only available in Claude Code. Skip on Claude.ai.

Blind comparison: Requires subagents. Skip.

Packaging: The package_skill.py script works anywhere with Python and a filesystem.

Updating existing skills: Preserve the original name. Copy to a writeable location before editing. If packaging manually, stage in /tmp/ first.

Cowork

Subagents available — full workflow works (spawn test cases in parallel, run baselines, grade, etc.). If timeouts occur, run in series.
No browser/display — use --static <output_path> for eval viewer. User clicks to open HTML, feedback downloads as feedback.json.
GENERATE THE EVAL VIEWER BEFORE evaluating inputs yourself. Get results in front of the human ASAP.
Description optimization (run_loop.py / run_eval.py) should work via claude -p.
When updating existing skills, follow the update guidance from the Claude.ai section.

Reference Files

The references/ directory has documentation loaded as needed:

references/schemas.md — JSON structures for evals.json, grading.json, benchmark.json, etc.
references/categories-and-patterns.md — Skill categories and 5 design patterns with templates
references/troubleshooting.md — Diagnosis and fixes for common skill issues

The agents/ directory contains instructions for specialized subagents:

agents/grader.md — How to evaluate assertions against outputs
agents/comparator.md — How to do blind A/B comparison between two outputs
agents/analyzer.md — How to analyze why one version beat another

Core Loop Reminder

Repeating for emphasis:

Figure out what the skill is about (classify category, define use cases, set success criteria)
Draft or edit the skill (use the right pattern for the category)
Run claude-with-access-to-the-skill on test prompts
With the user, evaluate the outputs:
- Create benchmark.json and run eval-viewer/generate_review.py to help the user review
- Run quantitative evals
Repeat until you and the user are satisfied
Run the pre-flight checklist
Package the final skill and return it to the user

Please add steps to your TodoList, if you have such a thing. If in Cowork, specifically put "Create evals JSON and run eval-viewer/generate_review.py so human can review test cases" in your TodoList.

Good luck!

Créateur de compétences

Recommandé pour