Automated summary
This skill runs automated QA sessions against the Grafema VS Code extension by driving code-server through Playwright and cross-checking every panel against graph data using MCP tools and the CLI.
Strengths
- Four explicit modes: auto-resume, specific file, re-validate open bugs (--recheck), and free-text custom task
- Full persistence in _qa/qa-state.json: per-line progress, bug registry, blocking gaps, task history and coverage stats
- Systematic cross-validation between what the agent sees on screen (multimodal screenshots) and the Grafema graph data
- Produces structured, timestamped artifacts (bug/gap reports, session reports, screenshots) that are directly actionable
Limitations
- Requires a heavy setup: a code-server Docker container on localhost:8080, Playwright chromium, and a populated Grafema graph database
- Tightly coupled to the Grafema project (paths under _qa/, Orchestrator.ts, file priority rules) and not reusable as-is elsewhere
- Capped at 10 bugs per session, so large scopes need several passes
Use it when you want to automatically verify that Grafema extension panels match the underlying graph data, or to re-validate bugs after a fix.
Skip it when the code-server container, Playwright or the Grafema database are unavailable, or when testing a project unrelated to this extension.
Security analysis
CautionThis SKILL.md contains no dangerous or exfiltrating instructions itself; it orchestrates a legitimate QA workflow. It is rated 'caution' because it exercises powerful tooling—Docker, browser automation, Bash, and MCP/CLI calls—and because its real behavior is delegated to a referenced agent file that a reviewer cannot see here, so the actual executed instructions are not fully auditable from this document alone.
- •Requires a running Docker container (docker-compose up -d) and interacts with it
- •Runs browser automation (Playwright/Chromium) via inline Node scripts in Bash
- •Delegates behavior to an external, unaudited agent instruction file (.claude/agents/qa-agent.md) whose contents are not part of this SKILL.md
- •Reads and writes state/report files under _qa/ (limited to project scope)
- •No destructive commands, no credential/token exfiltration, no obfuscated payloads, and network access appears limited to localhost:8080
Examples
/qa/qa packages/vscode/src/Orchestrator.ts/qa --recheckname: qa description: | Run automated QA checks on Grafema VS Code extension via Playwright + MCP cross-validation. Use when: (1) user says "/qa" to start or resume QA session, (2) user wants to validate extension panels against graph data, (3) user wants to re-check previously found bugs after a fix, (4) user wants to run a custom QA task. Requires Docker code-server running. author: Claude Code version: 1.0.0 date: 2026-02-20
/qa -- Grafema VS Code Extension QA
When to Use
- Validate VS Code extension UI against graph data
- Check specific files or resume from last session
- Re-check open bugs after fixes (version validation)
- Run custom QA tasks (e.g., "check hover tooltips only")
Usage
/qa # Auto-resume from last session
/qa packages/vscode/src/Orchestrator.ts # Check specific file
/qa --recheck # Re-validate all open bugs
/qa check only the Callers panel # Custom task (free text)
Prerequisites
- Docker container
code-serverrunning (cd demo && docker-compose up -d) - Playwright chromium available (auto-installed if missing)
- Code-server accessible at
http://localhost:8080 - Grafema graph database populated (extension must be analyzing a project)
How It Works
1. Parse Arguments
Determine the mode from the user's input:
- No args / empty: auto-resume. Find file with
status: "in-progress"in state, resume fromlastCheckedLine + 1. If none, pick next unchecked file. - File path: check that specific file from line 1 (or resume if already in-progress).
--recheck: re-validate all bugs withstatus: "open"and gaps withstatus: "blocking".- Anything else: treat as free-text custom task. Record in
customTasksregistry.
2. Load State
Read _qa/qa-state.json. This tracks:
- Per-file progress (which line we left off at)
- Bug registry (all bugs found across sessions)
- Gap registry (infrastructure gaps that block files)
- Custom task history
- Coverage statistics
- Version history
3. Check Docker
Verify the code-server Docker container is running and accessible. If not, print the start command and stop.
4. Launch QA Agent
Read the agent instructions from .claude/agents/qa-agent.md. The agent:
- Drives code-server via Playwright (inline Node.js scripts in Bash)
- Takes screenshots and reads them (multimodal -- Claude sees images)
- Cross-validates every panel with Grafema MCP tools and CLI
- Records bugs and gaps with full evidence
- Updates state after every checked line
- Stops after 10 bugs per session
5. Return Summary
After the session completes, report:
- How many entities were checked
- How many bugs and gaps were found
- Which reports were written
- What to do next (fix bugs, resolve gaps, re-run /qa)
Modes
Auto-Resume (default)
Finds the first file with status: "in-progress" and resumes from lastCheckedLine + 1. If no file is in progress, picks the next unchecked file using priority order: Orchestrator.ts first, then largest files first.
Specific File
Checks the given file from line 1. If the file was previously in-progress, resumes from where it left off.
Recheck
Re-validates all open bugs and blocking gaps. Does NOT check new lines. Marks fixed bugs as "fixed" and resolved gaps as "resolved". Useful after deploying a fix to verify it works.
Custom Task
Any argument that is not a file path and not --recheck is treated as a free-text instruction. The agent executes the task, records it in customTasks, and still logs any bugs/gaps found to the normal registries.
Output Locations
| Artifact | Path |
|----------|------|
| State file | _qa/qa-state.json |
| Bug reports | _qa/reports/bug-NNN.md |
| Gap reports | _qa/reports/gap-NNN.md |
| Session reports | _qa/reports/session-YYYY-MM-DD-HH-MM-SS.md |
| Screenshots | _qa/screenshots/ (gitignored) |
Session Limits
The agent stops after finding 10 bugs in a single session. This keeps reports manageable and encourages incremental fixing. Fix the bugs, then re-run /qa to continue.
Extension Panels
The extension registers 7 panels: Status, Value Trace, Callers, Blast Radius, Issues, Explorer, Debug Log. Plus the built-in Monaco hover tooltip.
Note: The panel formerly called "Edges Explorer" in early docs is actually "Blast Radius".
Bug vs Gap
- Bug (
ui-bugorcore-bug): single entity shows wrong/missing data in one panel. Recorded, agent continues. - Gap (
infrastructure-gap): entire panel broken across 5+ entities in multiple files. Blocks affected files, agent stops session.
Troubleshooting
Docker not running
cd demo && docker-compose up -d
Playwright not installed
The agent auto-installs chromium. If it fails:
npx playwright install chromium
Code-server not responding
Check Docker logs:
docker logs code-server
Panels not updating after click
The agent waits 3 seconds for async panel updates. If panels are still empty, this may indicate an infrastructure gap rather than a timing issue.
Screenshots unreadable
Small text in screenshots may be hard to read. The agent uses MCP/CLI for exact data validation and screenshots only for structural checks (panel visible, non-empty, has sections).
From the file to a first result
The download contains exactly the content displayed on this page. This copy may differ from the latest version in the source repository.
View the source and supporting filesThis download contains SKILL.md only. Any scripts, references, templates and dependencies are not included. Get the required files from the source, preserving their directory structure, before continuing.
1. Download the file
Download SKILL.mdFrom the root of a test project, save the file as .skill-downloads/automated-qa-checks-mrx3xw5o/SKILL.md, or use the command below. This folder is for inspecting the skill before installation.
Command for macOS/Linux or WSL: requires a POSIX shell and curl. On Windows without WSL, use the download button and create the folder manually.
mkdir -p '.skill-downloads/automated-qa-checks-mrx3xw5o' && curl --fail --show-error --location 'https://skills-guides.com/api/skills/automated-qa-checks-mrx3xw5o/download' --output '.skill-downloads/automated-qa-checks-mrx3xw5o/SKILL.md.part' && mv '.skill-downloads/automated-qa-checks-mrx3xw5o/SKILL.md.part' '.skill-downloads/automated-qa-checks-mrx3xw5o/SKILL.md'2. Check prerequisites
Read SKILL.md before using it. If it references scripts or external services, inspect them and prepare the required access in your test environment.
Original excerpt: Prerequisites
- Docker container
code-serverrunning (cd demo && docker-compose up -d) - Playwright chromium available (auto-installed if missing)
- Code-server accessible at
http://localhost:8080 - Grafema graph database populated (extension must be analyzing a project)
The platforms on this page are catalog indications. The paths below follow tool documentation; they do not prove this skill has been run in those tools. No minimum tool version is confirmed for this listing.
3. Load the skill
After inspection, copy the complete folder into your project using one of the commands below, then open a new tool session. The command stops if a folder with the same name already exists: compare the files before replacing it.
Claude Code
test -f '.skill-downloads/automated-qa-checks-mrx3xw5o/SKILL.md' && test -s '.skill-downloads/automated-qa-checks-mrx3xw5o/SKILL.md' && mkdir -p '.claude/skills' && test ! -e '.claude/skills/qa' && cp -R '.skill-downloads/automated-qa-checks-mrx3xw5o' '.claude/skills/qa'OpenSkills
OpenSkills must already be installed. The local install/read procedure is checked with OpenSkills 1.5.0, which requires Node.js ≥ 20.6.0. Catalog slugs are not resolved automatically.
Official OpenSkills installationopenskills install './.skill-downloads/automated-qa-checks-mrx3xw5o'openskills read 'automated-qa-checks-mrx3xw5o'Reading should display the installed file content and its base directory. It does not run the skill or install its dependencies.
For an agent that reads AGENTS.md, then sync the index. This command changes AGENTS.md: review the diff before using it.
openskills syncManual method or another tool: attach SKILL.md to a conversation, or open it in an agent that can read files. Include any required supporting resources. No universal folder is recognized by every tool.
4. Try a first task
Use a short, synthetic input: an API endpoint, a small document or an interface brief. In a new conversation, attach the file or give the agent its path, then adapt this request:
Read .skill-downloads/automated-qa-checks-mrx3xw5o/SKILL.md and apply its instructions to this example: [describe a test input]. Identify any missing files, dependencies or access. Produce the result requested by the skill without inventing data or running unreviewed scripts.Expected result: output matching the skill instructions, or a precise list of missing prerequisites. A read confirmation alone does not validate a first use. If an external resource is unavailable, that path remains untested.
ClickHouse Performance Comparison
Testing
Evaluate ClickHouse performance test results, check PR performance changes, compare with history, and assess whether a result is actionable.
Analyze CI Performance Reports
Testing
Analyze CI performance comparison reports for a ClickHouse PR. Cross-references master history to distinguish real regressions from flaky tests.
Santa Method
Testing
Multi-agent adversarial verification with two independent reviewers. Output is checked twice and iteratively fixed until both pass before shipping.