Automated summary
Verifies that a mock for a silicon API in tt-emule is complete by walking a checklist covering signature match, math correctness, format dispatch, JIT-cache freshness, and regression baselines.
Strengths
- Catches silent signature drift
- Prevents no-op stubs from passing unnoticed
- Ensures format-aware dispatch for different data types
- Validates regression baselines automatically
Limitations
- Requires manual wiping of JIT temporary directories
- Only applicable within the tt-emule ecosystem
- Depends on architecture-specific hardware specs
After implementing a mock for a silicon API in tt-emule, before declaring it done.
For non-emule or non-silicon-mocking tasks, or when no regression baseline exists.
Security analysis
SafeThe skill is a verification checklist for development workflows, executing only safe local commands (cmake build, rm -rf of JIT temp dirs, running regression scripts). No external data exfiltration, destructive actions beyond intended cleanup, or obfuscation.
No concerns found
How this analysis is madeExamples
I just finished implementing a mock for `typecast` in tt-emule. Run /verify-mock to check everything is correct before I ship.I'm in /compute-llk-bringup triaging a PCC failure. The issue might be a bad mock. Run /verify-mock on the `compute_reduce` mock to confirm.name: verify-mock description: Verification checklist to walk before declaring a tt-emule mock complete. Use after implementing a stub (typically at the end of /implement-mock or /compute-llk-bringup) to catch the recurring failure modes — signature drift, no-op math, missing format dispatch, JIT-cache staleness, regression baseline drift. user_invocable: true
/verify-mock — Verify your mock before declaring done
You've implemented a mock for a silicon API in tt-emule. Before declaring it complete, walk this checklist. Skipping any step is how regressions creep in.
Invoke from /implement-mock Step 4 or /compute-llk-bringup PCC
triage, or directly when finishing up a mock.
Pre-implementation
- Confirm you ran
/arch-lookup "<silicon_function>"and have the authoritative HW spec. - Confirm you picked a Strategy (A/B/C — see
/implement-mockStep 2) and which kernel-API layer (layer-1 / 1.5 / 2 / 3, perdocs/kernel-api-layers.md) the mock targets. - If the spec is arch-specific (WH vs BH vs QSR), confirm which arch
the target test uses (
TT_EMULE_ARCH=blackhole|wormhole|quasar). - Check no existing mock already covers the same surface (avoid duplicates / conflicting overloads).
Implementation
- The mock signature exactly matches silicon's signature (template params, defaults, return type). Mismatches cause silent ADL surprises.
- Real math (not a no-op) if the test will check PCC against torch. No-op stubs are only OK for HW pipeline state (UNPACK/MATH/PACK config calls).
- Format-aware path for bf16 vs fp32 vs uint16 vs Bfp8_b vs Bfp4_b if
the API reads/writes a CB. Use the enum-driven predicates
(
__emule_compute::cb_is_32bit_format(cb_id)and siblings) — never page-size heuristics (seedocs/cb-dataformat.md). - Use
__emule_nfaces::rowmajor_to_nfaces[]on CB reads/writes — emule DST is row-major, CB tiles are face-packed. - If touching DST: call
__emule_dst_check(idst, "name")early and__emule_dst_mark_dirty(idst)on writes. - If the silicon impl has multiple template variants (e.g.
<BroadcastType bcast_type>), all variants are supported (useif constexprto dispatch). - Comment cites silicon source (DeepWiki page, Confluence page, or
tt_llk_*/...:line) for non-obvious choices.
Build + sanity
- Build passes (only when you touched
emulated_program_runner.cppor other runner sources):cmake --build ${TT_METAL_DIR}/build_emule -j$(nproc) - Wipe per-test JIT temp dirs before testing:
The persistent disk cache atrm -rf /tmp/tt_emule_jit_* /tmp/tt_emule_src_*/tmp/tt_emule_jit_cache_*is hashed and self-invalidates on header content change; per-test temp dirs at/tmp/tt_emule_jit_*/need an explicit wipe.
Verification
- Sentinel test passes — your project's smallest end-to-end emule test that exercises the JIT compile + dispatch path.
- Target test reaches the new mock (verify with
TT_EMULE_KEEP_JIT_TEMP=1+ grep the patchedkernel.cpp/wrapper.cppunder/tmp/tt_emule_jit_*/). - Target test passes (or PCC improves above its threshold).
- tt-metal regression matches the recorded baseline:
Run the three sequentially (shared JIT cache). Any test that previously passed and now fails blocks ship; consultTT_METAL_DIR=<tt-metal-checkout> bash scripts/run_regression_wormhole.sh 2>&1 | tee /tmp/rg-wh.log TT_METAL_DIR=<tt-metal-checkout> bash scripts/run_regression_blackhole.sh 2>&1 | tee /tmp/rg-bh.log TT_METAL_DIR=<tt-metal-checkout> bash scripts/run_regression_quasar.sh 2>&1 | tee /tmp/rg-qs.log.github/known-failures-quasar.txtfor the QS allowlist. - Coverage came from an existing tt-metal test, not a new one.
Prefer fixing the mock so a canonical tt-metal test passes over
authoring a test. A new tt-metal test is a last resort; when
genuinely unavoidable it must live under
tests/emule/(e.g.tests/emule/ccl/), formatted to tt-metal's pre-commit config (black line-length 120, isort, autoflake), wired into the emule pytest runner (scripts/run_ttnn_pytests_*.sh), with the pin bumped to the companion commit. See the CLAUDE.md project rule.
Documentation
- If you added a new strategy / pattern, append to
.claude/references/emule-mapping.md(the catalog) and link from/implement-mockStep 2 (the strategy taxonomy). - If you discovered a HW-spec-vs-emule-mock divergence the test
doesn't catch, note it in the commit message and in any relevant
docs/<subsystem>-emulation.md.
Common gotchas
- No-op stub compiles but produces zero/garbage at runtime. Add
real math even if "obvious" —
recip_tilereturning 0 is harder to debug thanrecip_tileundefined. - Template parameter mismatch with silicon signature. Causes "no matching function" errors that look like the mock isn't found. Always copy the silicon signature exactly, then maybe add defaults.
#ifdef __EMULE_JIT_MODEplaced inside a#if defined(COMPILE_FOR_*)block. Both run; the inner one wins. Make sure the gate is at the right level — usually OUTSIDE the per-RISC guard.- Allowlist-add poisons sentinel. If the op's op.hpp doesn't compile under TRISC (even if your test only uses BRISC), the whole allowlist breaks. The sentinel run catches this.
tt_l1_ptr,VALID/INVALIDundefined. These come fromdataflow_api.h(tt_l1_ptr) andhostdevcommon/common_values.hpp(VALID/INVALID). Both must be#included for any code path that uses them — silicon path may pull them transitively, emule may not.compute_kernel_hw_startupredefinition. Bothjit_kernel_stubs.hppandapi/compute/compute_kernel_hw_startup.hdefine overloads. Use the__EMULE_COMPUTE_KERNEL_HW_STARTUP_DEFINEDguard pattern (first-included-wins).
Anti-checklist
These are NOT done criteria — don't get hung up:
- Bit-exact match with silicon output. PCC > test's threshold (~0.998) is the bar.
- Real
sfpi::SIMD math (the shim provides types only). - Fabric / multichip semantics (out of scope).
Related skills
/implement-mock— end-to-end workflow (this checklist is its Step 4 in expanded form)./compute-llk-bringup— specialization for LLK compute shims./memory-debug— when verification fails with PCC < threshold or ATOL mismatch (partial zeros, off-by-N writes).
From the file to a first result
The download contains exactly the content displayed on this page. This copy may differ from the latest version in the source repository.
View the source and supporting filesThis download contains SKILL.md only. Any scripts, references, templates and dependencies are not included. Get the required files from the source, preserving their directory structure, before continuing.
1. Download the file
Download SKILL.mdFrom the root of a test project, save the file as .skill-downloads/verify-tt-emule-mock-mrx3xwfa/SKILL.md, or use the command below. This folder is for inspecting the skill before installation.
Command for macOS/Linux or WSL: requires a POSIX shell and curl. On Windows without WSL, use the download button and create the folder manually.
mkdir -p '.skill-downloads/verify-tt-emule-mock-mrx3xwfa' && curl --fail --show-error --location 'https://skills-guides.com/api/skills/verify-tt-emule-mock-mrx3xwfa/download' --output '.skill-downloads/verify-tt-emule-mock-mrx3xwfa/SKILL.md.part' && mv '.skill-downloads/verify-tt-emule-mock-mrx3xwfa/SKILL.md.part' '.skill-downloads/verify-tt-emule-mock-mrx3xwfa/SKILL.md'2. Check prerequisites
Read SKILL.md before using it. If it references scripts or external services, inspect them and prepare the required access in your test environment.
No recognized prerequisites section in this file. This does not mean it is self-contained: libraries, tools and access requirements may be mentioned elsewhere in the content or source repository.
The platforms on this page are catalog indications. The paths below follow tool documentation; they do not prove this skill has been run in those tools. No minimum tool version is confirmed for this listing.
3. Load the skill
After inspection, copy the complete folder into your project using one of the commands below, then open a new tool session. The command stops if a folder with the same name already exists: compare the files before replacing it.
Claude Code
test -f '.skill-downloads/verify-tt-emule-mock-mrx3xwfa/SKILL.md' && test -s '.skill-downloads/verify-tt-emule-mock-mrx3xwfa/SKILL.md' && mkdir -p '.claude/skills' && test ! -e '.claude/skills/verify-mock' && cp -R '.skill-downloads/verify-tt-emule-mock-mrx3xwfa' '.claude/skills/verify-mock'OpenSkills
OpenSkills must already be installed. The local install/read procedure is checked with OpenSkills 1.5.0, which requires Node.js ≥ 20.6.0. Catalog slugs are not resolved automatically.
Official OpenSkills installationopenskills install './.skill-downloads/verify-tt-emule-mock-mrx3xwfa'openskills read 'verify-tt-emule-mock-mrx3xwfa'Reading should display the installed file content and its base directory. It does not run the skill or install its dependencies.
For an agent that reads AGENTS.md, then sync the index. This command changes AGENTS.md: review the diff before using it.
openskills syncManual method or another tool: attach SKILL.md to a conversation, or open it in an agent that can read files. Include any required supporting resources. No universal folder is recognized by every tool.
4. Try a first task
Use a short, synthetic input: an API endpoint, a small document or an interface brief. In a new conversation, attach the file or give the agent its path, then adapt this request:
Read .skill-downloads/verify-tt-emule-mock-mrx3xwfa/SKILL.md and apply its instructions to this example: [describe a test input]. Identify any missing files, dependencies or access. Produce the result requested by the skill without inventing data or running unreviewed scripts.Expected result: output matching the skill instructions, or a precise list of missing prerequisites. A read confirmation alone does not validate a first use. If an external resource is unavailable, that path remains untested.
ClickHouse Performance Comparison
Testing
Evaluate ClickHouse performance test results, check PR performance changes, compare with history, and assess whether a result is actionable.
Analyze CI Performance Reports
Testing
Analyze CI performance comparison reports for a ClickHouse PR. Cross-references master history to distinguish real regressions from flaky tests.
Santa Method
Testing
Multi-agent adversarial verification with two independent reviewers. Output is checked twice and iteratively fixed until both pass before shipping.