Smart Search with BM25 Ranking

VerifiedCaution

Smart search with BM25 ranking. Queries the graph index first for relevance-ranked results, then falls back to grep for full-text content search. Supports filters like type, status, priority, and backlinks, plus shortcuts. Best for quickly finding notes in large Obsidian vaults.

Sby Skills Guide Bot
ProductivityIntermediate
6/2/2026
Claude Code
#search#bm25#graph-index#grep#note-taking

Recommended for

Automated summary

Smart search across notes using BM25 relevance ranking, querying the graph index first with ranked results then falling back to grep for content search.

Strengths

  • BM25 relevance ranking for accurate results.
  • Fast initial search via graph index (~50ms).
  • Automatic fallback to grep for full-text search.
  • Supports filters by type, status, priority, and backlinks.

Limitations

  • Requires a pre-built graph index.
  • Grep search can be slow on large vaults (2-5s).
  • Does not handle complex multi-filter advanced queries.
When to use it

Use this command when you need to quickly find a specific note among many, benefiting from relevance-ranked results.

When not to use it

Avoid this command for very complex searches requiring advanced boolean logic or unsupported regex patterns.

Security analysis

Caution
Quality score85/100

The skill uses bash commands with unescaped user input, posing a medium risk of command injection in the agent's execution environment. No network exfiltration or obfuscated payloads, but the potential for destructive commands exists.

Findings
  • •Command injection vulnerability: User-supplied query is interpolated directly into shell commands (graph-query.js --search and grep -r) without sanitization, allowing arbitrary command execution if special characters are included.
How this analysis is made

Examples

Simple keyword search
/search kafka
Filtered search by type and status
/search type:Adr status:proposed
Backlink search
/search backlinks:Project - MyProject

context: fork

/search

Smart search using BM25 relevance ranking. Queries the graph index first with ranked results, then falls back to grep for content search.

Usage

/search kafka
/search "API gateway"
/search type:Adr status:proposed
/search backlinks:Project - MyProject

Instructions

When the user invokes /search <query>:

Phase 1: Parse Query Type

Determine what kind of search this is:

| Pattern | Search Type | Action | |---------|-------------|--------| | type:<Type> | Type filter | Use graph --type | | status:<status> | Status filter | Use graph --status | | priority:<priority> | Priority filter | Use graph --priority | | backlinks:<note> | Backlink search | Use graph --backlinks | | Simple keyword | Keyword search | Use graph --search, then grep | | Regex pattern | Content search | Skip to grep |

Phase 2: Query Graph Index First

Always start with the graph index (unless it's a regex):

# Check index exists
if [ ! -f ".graph/index.json" ]; then
  npm run graph:build
fi

# Run graph query
node scripts/graph-query.js --search "<query>"

Phase 3: Evaluate Results

After running the graph query:

  1. If results found: Present them formatted as a table
  2. If no results OR user needs content search: Fall back to grep
# Grep fallback for content search
grep -r "<query>" --include="*.md" --exclude-dir=".git" --exclude-dir=".obsidian" --exclude-dir="node_modules" --exclude-dir=".smart-env"

Phase 4: Present Combined Results

Format output with BM25 relevance scores:

## Search Results for "<query>"

### Graph Index Results (BM25 ranked)
Found **X** notes matching in graph index:

| Score | Type | Title |
|-------|------|-------|
| 14.65 | Page | Kafka to SAP Integration |
| 11.32 | Adr | ADR - Kafka Integration |
| 9.45 | Meeting | Data Platform Discussion |

*Results ranked by relevance (higher score = better match)*

### Content Search Results (full-text)
Found **Y** additional matches in note content:

- `Page - Kafka Architecture.md:45` - "...kafka cluster configuration..."
- `Meeting - 2025-01-10 Data Team.md:12` - "...discussed kafka..."

Query Shortcuts

| Shortcut | Expands To | |----------|------------| | /search t:Adr | --type Adr | | /search s:active | --status active | | /search p:high | --priority high | | /search b:Note | --backlinks "Note" | | /search orphans | --orphans | | /search broken | --broken-links | | /search stale | --stale |

Examples

/search kafka                    # Keyword in graph + content
/search type:Meeting kafka       # Meetings mentioning kafka
/search backlinks:Project - MyProject # Notes linking to project
/search status:proposed          # All proposed items
/search orphans                  # Orphaned notes
/search "event.*driven"          # Regex - grep only

Performance Notes

  • Graph search (BM25): ~50ms for 1500 notes (pre-indexed with ranking)
  • Grep search: ~2-5s depending on vault size
  • Combined: Graph first catches 80%+ of searches instantly
  • Relevance ranking: Results are sorted by BM25 score, so top results are most relevant
  • IDF weighting: Rare terms like "kafka" score higher than common terms like "the"

Related

  • /graph-query - Direct graph queries with more options
  • /orphans - Dedicated orphan analysis
  • /broken-links - Dedicated broken link analysis

From the file to a first result

The download contains exactly the content displayed on this page. This copy may differ from the latest version in the source repository.

View the source and supporting files

This download contains SKILL.md only. Any scripts, references, templates and dependencies are not included. Get the required files from the source, preserving their directory structure, before continuing.

1. Download the file

Download SKILL.md

From the root of a test project, save the file as .skill-downloads/smart-search-with-bm25-ranking-mm5t8c44/SKILL.md, or use the command below. This folder is for inspecting the skill before installation.

Command for macOS/Linux or WSL: requires a POSIX shell and curl. On Windows without WSL, use the download button and create the folder manually.

mkdir -p '.skill-downloads/smart-search-with-bm25-ranking-mm5t8c44' && curl --fail --show-error --location 'https://skills-guides.com/api/skills/smart-search-with-bm25-ranking-mm5t8c44/download' --output '.skill-downloads/smart-search-with-bm25-ranking-mm5t8c44/SKILL.md.part' && mv '.skill-downloads/smart-search-with-bm25-ranking-mm5t8c44/SKILL.md.part' '.skill-downloads/smart-search-with-bm25-ranking-mm5t8c44/SKILL.md'

2. Check prerequisites

Read SKILL.md before using it. If it references scripts or external services, inspect them and prepare the required access in your test environment.

No recognized prerequisites section in this file. This does not mean it is self-contained: libraries, tools and access requirements may be mentioned elsewhere in the content or source repository.

The platforms on this page are catalog indications. The paths below follow tool documentation; they do not prove this skill has been run in those tools. No minimum tool version is confirmed for this listing.

3. Load the skill

This file does not provide a usable name/description header for this native procedure. It remains available to download and inspect; use the manual method below.

Manual method or another tool: attach SKILL.md to a conversation, or open it in an agent that can read files. Include any required supporting resources. No universal folder is recognized by every tool.

4. Try a first task

Use a short, synthetic input: an API endpoint, a small document or an interface brief. In a new conversation, attach the file or give the agent its path, then adapt this request:

Read .skill-downloads/smart-search-with-bm25-ranking-mm5t8c44/SKILL.md and apply its instructions to this example: [describe a test input]. Identify any missing files, dependencies or access. Produce the result requested by the skill without inventing data or running unreviewed scripts.

Expected result: output matching the skill instructions, or a precise list of missing prerequisites. A read confirmation alone does not validate a first use. If an external resource is unavailable, that path remains untested.

Related skills