Firecrawl CLI - Web Scraping & Search

VerifiedCaution

Web scraping, search, crawling, and browser interaction via Firecrawl CLI. Use for fetching URLs, scraping content, extracting structured data, and interacting with JavaScript-heavy pages.

Sby Skills Guide Bot
Data & AIIntermediate
7/22/2026
Claude CodeCursorCodex
#web-scraping#crawling#search#data-extraction#cli

Recommended for

Automated summary

This skill is a condensed Firecrawl CLI reference for searching the web, fetching and scraping URLs, mapping or crawling sites, and extracting structured data.

Strengths

  • Lays out a clear search → scrape → map → crawl → interact escalation path so heavier operations are only used when needed
  • Enforces writing output to .firecrawl/ with -o to keep context bloat down
  • Covers structured extraction via --format json --schema and JS-heavy pages via --wait-for
  • Explains when to try WebFetch first (free) before spending Firecrawl credits

Limitations

  • Requires a global install (npm install -g firecrawl-cli) plus authentication and Firecrawl credits
  • Advanced capabilities (full crawl, AI agent, interact, browser profiles) are only sketched and defer to separate skills to install
  • Each scrape or targeted query burns credits, and the skill only partially quantifies that cost
When to use it

Use it when a task needs web search, full-page content extraction, or site traversal beyond what WebFetch can return.

When not to use it

Avoid it for reading local files, git operations, or code editing, and when WebFetch already handles a simple static page.

Security analysis

Caution
Quality score85/100

The skill is a straightforward CLI reference for a legitimate third-party scraping service, and the declared allowed-tools scope it narrowly to `firecrawl`/`npx firecrawl` invocations. There is no obfuscation, no curl|sh, no destructive file operations, and no attempt to exfiltrate secrets; the only powerful aspects are network access, a global npm install, credential handling via the CLI's own auth, and remote browser code execution, all of which are inherent to the tool's stated purpose.

Findings
  • •Instructs a global npm install (`npm install -g firecrawl-cli`) which executes third-party package code and is a supply-chain consideration
  • •`firecrawl auth` / `firecrawl login` will handle API credentials; users should be aware tokens are stored by the CLI
  • •`firecrawl browser execute --code "..."` runs arbitrary JavaScript in a remote browser session, a powerful capability
  • •All commands send URLs/page content to the Firecrawl third-party service (intended purpose, but data leaves the machine)
  • •`firecrawl download -y` writes many files locally under `.firecrawl/` (benign but bulk disk writes)
How this analysis is made

Examples

Web search with content
Search the web for recent benchmarks comparing vector databases and scrape the top 5 results into .firecrawl/results.json.
Clean article scrape
Scrape https://example.com/blog/launch-post, strip the nav and footer, and save the markdown to .firecrawl/page.md.
Structured pricing extraction
Scrape https://example.com/pricing with a JSON schema for plan name and monthly price, and save it to .firecrawl/data.json.

name: firecrawl-lean description: | Web scraping, search, crawling, and browser interaction via Firecrawl CLI. Use when user wants to search the web, fetch a URL, scrape content, crawl docs, download a site, extract structured data, or interact with JS-heavy/login-gated pages. Triggers on: "search for", "scrape", "fetch this URL", "get the page", "crawl", "download the site", "extract structured data", "click the button", "fill the form", "log in to". Do NOT use for local files, git, or code editing. For advanced use (bulk crawl, AI agent extraction, browser profiles, parallel jobs): install firecrawl-crawl, firecrawl-agent, or firecrawl-instruct skills. allowed-tools:

  • Bash(firecrawl *)
  • Bash(npx firecrawl *)

Firecrawl CLI — lean reference

Check status: firecrawl --status (shows auth + credits + concurrency limit). Not installed? Run: npm install -g firecrawl-cli && firecrawl auth Auth commands: firecrawl login, firecrawl logout, firecrawl view-config

WebFetch vs Firecrawl

Try WebFetch first (free). Use Firecrawl when:

  • Need web search (WebFetch can't search)
  • JS-heavy SPA / marketing pages
  • Full article content (WebFetch truncates long pages)
  • WebFetch output < 1000 chars or looks incomplete

Escalation pattern

search → scrape → map+scrape → crawl* → interact*

* = advanced, install dedicated skill if needed

Commands

Search — no URL yet, find pages:

firecrawl search "query" --limit 5 -o .firecrawl/results.json --json
firecrawl search "query" --scrape -o .firecrawl/results.json --json        # include full content
firecrawl search "query" --sources news --tbs qdr:d --json                 # news, past day
firecrawl search "query" --country US --location "New York" --json         # geo-targeted

Scrape — have a URL:

firecrawl scrape "<url>" -o .firecrawl/page.md
firecrawl scrape "<url>" --only-main-content -o .firecrawl/page.md        # strip nav/footer
firecrawl scrape "<url>" --wait-for 3000 -o .firecrawl/page.md            # wait for JS
firecrawl scrape "<url>" --query "What is the enterprise price?"            # targeted Q (5 credits)
firecrawl scrape "<url>" --format markdown,html,screenshot -o .firecrawl/page.md  # multi-format
firecrawl scrape "<url>" --format json --schema '{"price":"string"}' -o .firecrawl/data.json  # structured

Map — find URLs within a site:

firecrawl map "<url>" --search "authentication" -o .firecrawl/urls.txt
firecrawl map "<url>" --limit 200 --json -o .firecrawl/urls.json
firecrawl map "<url>" --include-subdomains --sitemap -o .firecrawl/urls.json

Crawl — bulk extract site section (advanced — install firecrawl-crawl for full docs):

firecrawl crawl "<url>" --include-paths /docs --limit 50 --wait -o .firecrawl/crawl.json
firecrawl crawl "<url>" --max-depth 3 --delay 500 --exclude-paths /blog --wait -o .firecrawl/crawl.json

Download — save site as local files (advanced):

firecrawl download "<url>" --limit 20 -y -o .firecrawl/site/

Interact — click/fill forms after scrape (advanced — install firecrawl-instruct for full docs):

firecrawl scrape "<url>"
firecrawl interact --prompt "Click the login button"
firecrawl interact stop

Agent — AI-powered structured extraction (advanced — install firecrawl-agent for full docs):

firecrawl agent "extract all pricing tiers" --wait -o .firecrawl/data.json
firecrawl agent "extract pricing" --urls "<url>" --schema '{"tiers":[]}' --wait -o .firecrawl/data.json

Browser — CDP/Playwright-style automation (distinct from interact):

firecrawl browser launch-session "<url>"
firecrawl browser execute --code "document.querySelector('button').click()"
firecrawl browser list
firecrawl browser close <session-id>

Output rules

  • Always write to .firecrawl/ with -o to avoid context bloat
  • Add .firecrawl/ to .gitignore
  • Always quote URLs (? and & are shell special chars)
  • Never read entire output at once: head -50 file.md or grep "keyword" file.md
  • Naming: .firecrawl/search-{query}.json, .firecrawl/{site}-{path}.md

Extract URLs from JSON results

jq -r '.data.web[].url' .firecrawl/search.json

Credits

firecrawl credit-usage

From the file to a first result

The download contains exactly the content displayed on this page. This copy may differ from the latest version in the source repository.

View the source and supporting files

This download contains SKILL.md only. Any scripts, references, templates and dependencies are not included. Get the required files from the source, preserving their directory structure, before continuing.

1. Download the file

Download SKILL.md

From the root of a test project, save the file as .skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj/SKILL.md, or use the command below. This folder is for inspecting the skill before installation.

Command for macOS/Linux or WSL: requires a POSIX shell and curl. On Windows without WSL, use the download button and create the folder manually.

mkdir -p '.skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj' && curl --fail --show-error --location 'https://skills-guides.com/api/skills/firecrawl-cli-web-scraping-search-mrwakamj/download' --output '.skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj/SKILL.md.part' && mv '.skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj/SKILL.md.part' '.skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj/SKILL.md'

2. Check prerequisites

Read SKILL.md before using it. If it references scripts or external services, inspect them and prepare the required access in your test environment.

No recognized prerequisites section in this file. This does not mean it is self-contained: libraries, tools and access requirements may be mentioned elsewhere in the content or source repository.

The platforms on this page are catalog indications. The paths below follow tool documentation; they do not prove this skill has been run in those tools. No minimum tool version is confirmed for this listing.

3. Load the skill

After inspection, copy the complete folder into your project using one of the commands below, then open a new tool session. The command stops if a folder with the same name already exists: compare the files before replacing it.

Claude Code
test -f '.skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj/SKILL.md' && test -s '.skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj/SKILL.md' && mkdir -p '.claude/skills' && test ! -e '.claude/skills/firecrawl-lean' && cp -R '.skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj' '.claude/skills/firecrawl-lean'

Cursor
test -f '.skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj/SKILL.md' && test -s '.skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj/SKILL.md' && mkdir -p '.cursor/skills' && test ! -e '.cursor/skills/firecrawl-lean' && cp -R '.skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj' '.cursor/skills/firecrawl-lean'

Codex
test -f '.skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj/SKILL.md' && test -s '.skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj/SKILL.md' && mkdir -p '.agents/skills' && test ! -e '.agents/skills/firecrawl-lean' && cp -R '.skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj' '.agents/skills/firecrawl-lean'

OpenSkills

OpenSkills must already be installed. The local install/read procedure is checked with OpenSkills 1.5.0, which requires Node.js ≥ 20.6.0. Catalog slugs are not resolved automatically.

Official OpenSkills installation
openskills install './.skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj'

openskills read 'firecrawl-cli-web-scraping-search-mrwakamj'

Reading should display the installed file content and its base directory. It does not run the skill or install its dependencies.

For an agent that reads AGENTS.md, then sync the index. This command changes AGENTS.md: review the diff before using it.

openskills sync

Manual method or another tool: attach SKILL.md to a conversation, or open it in an agent that can read files. Include any required supporting resources. No universal folder is recognized by every tool.

4. Try a first task

Use a short, synthetic input: an API endpoint, a small document or an interface brief. In a new conversation, attach the file or give the agent its path, then adapt this request:

Read .skill-downloads/firecrawl-cli-web-scraping-search-mrwakamj/SKILL.md and apply its instructions to this example: [describe a test input]. Identify any missing files, dependencies or access. Produce the result requested by the skill without inventing data or running unreviewed scripts.

Expected result: output matching the skill instructions, or a precise list of missing prerequisites. A read confirmation alone does not validate a first use. If an external resource is unavailable, that path remains untested.

Related skills