Automated summary
Analyzes test failures during the verification phase and suggests automatic or manual fixes.
Strengths
- Follows a systematic research protocol to understand codebase context before analysis.
- Clearly distinguishes recoverable errors (typos, missing imports) from those requiring human intervention.
- Provides concrete, code-level fix suggestions.
- Integrates into a Feature Swarm development workflow.
Limitations
- Only handles simple recoverable errors; ignores architecture, environment, or dependency issues.
- Requires structured test output and access to source code to be effective.
- May miss complex logic errors or subtle side effects.
Use this skill when tests fail during the verification phase of a feature in a swarm-based development process.
Avoid using it for failures caused by infrastructure (database, network) or major architectural issues requiring redesign.
Security analysis
SafeThe skill exclusively uses read-only tools (Read, Glob, Grep) for codebase exploration and analysis. It outputs structured JSON without any destructive, exfiltrating, or obfuscated actions. No shell, network, or file modification commands are used or suggested.
No concerns found
How this analysis is madeExamples
Analyze this test failure: FAILED tests/test_user.py::test_create_user - NameError: name 'datetime' is not defined. The feature ID is USER-42 and issue number is #101.The verification phase is failing: FAILED tests/test_math.py::test_add - AssertionError: assert 3 == 5. Investigate and suggest a fix.name: feature-verifier description: > Analyze test failures and suggest fixes for Feature Swarm verification. Use when tests fail during the verification phase to diagnose root cause and determine if automatic recovery is possible. allowed-tools: Read,Glob,Grep
Verifier Agent - Failure Analysis
You are analyzing a test failure from the Feature Swarm verification phase. Your goal is to understand why tests failed and provide actionable guidance.
Phase 0: Research (MANDATORY - DO THIS FIRST)
<research_protocol>
Before analyzing failures, you MUST explore the codebase to understand context.
1. Find Relevant Files
Glob "swarm_attack/**/*.py"
Glob "tests/**/*.py"
2. Search for Error Context
Grep "class.*Error" swarm_attack/
Grep "def test_" tests/
3. Read Key Files
Read CLAUDE.md
Read swarm_attack/agents/base.py
4. Document Findings Before proceeding, note:
- [ ] Existing error handling patterns
- [ ] Test patterns in the codebase
- [ ] Related modules that might be affected
</research_protocol>
DO NOT analyze failures without understanding the codebase context first.
Instructions
- Analyze the test output - Identify which tests failed and why
- Determine root cause - What's the underlying issue?
- Assess recoverability - Can this be fixed automatically by retrying with CoderAgent?
- Suggest fixes - If recoverable, what specific changes are needed?
Recoverability Guidelines
Recoverable (can retry with CoderAgent):
- Missing import statements
- Typos in function/variable names
- Off-by-one errors
- Missing return statements
- Incorrect parameter order
- Simple logic errors
NOT Recoverable (needs human intervention):
- Fundamental architecture issues
- Missing dependencies/packages
- Environment configuration problems
- Test framework issues
- Circular dependencies
- External API failures
Output Format
You MUST respond with valid JSON in this exact format:
{
"root_cause": "Brief description of what went wrong",
"recoverable": true,
"suggested_fix": "Specific code changes needed to fix the issue",
"affected_files": ["path/to/file1.py", "path/to/file2.py"]
}
If not recoverable, set suggested_fix to null and explain in root_cause why human intervention is needed.
Examples
Example 1: Missing Import (Recoverable)
Test Output:
FAILED tests/test_user.py::test_create_user - NameError: name 'datetime' is not defined
Response:
{
"root_cause": "Missing import for datetime module in user.py",
"recoverable": true,
"suggested_fix": "Add 'from datetime import datetime' at the top of src/user.py",
"affected_files": ["src/user.py"]
}
Example 2: Logic Error (Recoverable)
Test Output:
FAILED tests/test_math.py::test_add - AssertionError: assert 3 == 5
where 3 = add(2, 3)
Response:
{
"root_cause": "add() function has incorrect implementation - returning a-b instead of a+b",
"recoverable": true,
"suggested_fix": "Change 'return a - b' to 'return a + b' in the add function",
"affected_files": ["src/math.py"]
}
Example 3: Architecture Issue (Not Recoverable)
Test Output:
FAILED tests/test_api.py::test_endpoint - ConnectionError: Database not configured
Response:
{
"root_cause": "Tests require database connection but no database is configured. This is an infrastructure/environment issue that cannot be fixed by code changes alone.",
"recoverable": false,
"suggested_fix": null,
"affected_files": []
}
Context Provided
You will receive:
- Test Output: The raw pytest output showing failures
- Feature ID: The feature being implemented
- Issue Number: The specific issue being worked on
Use the allowed tools (Read, Glob, Grep) to examine source files if needed to provide more accurate analysis.
From the file to a first result
The download contains exactly the content displayed on this page. This copy may differ from the latest version in the source repository.
View the source and supporting filesThis download contains SKILL.md only. Any scripts, references, templates and dependencies are not included. Get the required files from the source, preserving their directory structure, before continuing.
1. Download the file
Download SKILL.mdFrom the root of a test project, save the file as .skill-downloads/test-failure-verifier-mma3jryk/SKILL.md, or use the command below. This folder is for inspecting the skill before installation.
Command for macOS/Linux or WSL: requires a POSIX shell and curl. On Windows without WSL, use the download button and create the folder manually.
mkdir -p '.skill-downloads/test-failure-verifier-mma3jryk' && curl --fail --show-error --location 'https://skills-guides.com/api/skills/test-failure-verifier-mma3jryk/download' --output '.skill-downloads/test-failure-verifier-mma3jryk/SKILL.md.part' && mv '.skill-downloads/test-failure-verifier-mma3jryk/SKILL.md.part' '.skill-downloads/test-failure-verifier-mma3jryk/SKILL.md'2. Check prerequisites
Read SKILL.md before using it. If it references scripts or external services, inspect them and prepare the required access in your test environment.
Tools declared by the author (check availability in your agent): Read,Glob,Grep
No recognized prerequisites section in this file. This does not mean it is self-contained: libraries, tools and access requirements may be mentioned elsewhere in the content or source repository.
The platforms on this page are catalog indications. The paths below follow tool documentation; they do not prove this skill has been run in those tools. No minimum tool version is confirmed for this listing.
3. Load the skill
After inspection, copy the complete folder into your project using one of the commands below, then open a new tool session. The command stops if a folder with the same name already exists: compare the files before replacing it.
Claude Code
test -f '.skill-downloads/test-failure-verifier-mma3jryk/SKILL.md' && test -s '.skill-downloads/test-failure-verifier-mma3jryk/SKILL.md' && mkdir -p '.claude/skills' && test ! -e '.claude/skills/feature-verifier' && cp -R '.skill-downloads/test-failure-verifier-mma3jryk' '.claude/skills/feature-verifier'OpenSkills
OpenSkills must already be installed. The local install/read procedure is checked with OpenSkills 1.5.0, which requires Node.js ≥ 20.6.0. Catalog slugs are not resolved automatically.
Official OpenSkills installationopenskills install './.skill-downloads/test-failure-verifier-mma3jryk'openskills read 'test-failure-verifier-mma3jryk'Reading should display the installed file content and its base directory. It does not run the skill or install its dependencies.
For an agent that reads AGENTS.md, then sync the index. This command changes AGENTS.md: review the diff before using it.
openskills syncManual method or another tool: attach SKILL.md to a conversation, or open it in an agent that can read files. Include any required supporting resources. No universal folder is recognized by every tool.
4. Try a first task
Use a short, synthetic input: an API endpoint, a small document or an interface brief. In a new conversation, attach the file or give the agent its path, then adapt this request:
Read .skill-downloads/test-failure-verifier-mma3jryk/SKILL.md and apply its instructions to this example: [describe a test input]. Identify any missing files, dependencies or access. Produce the result requested by the skill without inventing data or running unreviewed scripts.Expected result: output matching the skill instructions, or a precise list of missing prerequisites. A read confirmation alone does not validate a first use. If an external resource is unavailable, that path remains untested.
ClickHouse Performance Comparison
Testing
Evaluate ClickHouse performance test results, check PR performance changes, compare with history, and assess whether a result is actionable.
Analyze CI Performance Reports
Testing
Analyze CI performance comparison reports for a ClickHouse PR. Cross-references master history to distinguish real regressions from flaky tests.
Santa Method
Testing
Multi-agent adversarial verification with two independent reviewers. Output is checked twice and iteratively fixed until both pass before shipping.