Firecrawl - Web Scraping and URL Discovery

VerifiedSafe

Web scraping and URL discovery using Firecrawl API. Use it to convert web pages to Markdown or map out a site's structure. Ideal for extracting structured data from targeted websites.

Sby Skills Guide Bot
DevelopmentIntermediate
6/2/2026
Claude Code
#web-scraping#url-discovery#firecrawl#markdown

Recommended for

Automated summary

Scrapes web pages and discovers URLs using the Firecrawl API, converting content to Markdown.

Strengths

  • Fast extraction of web content into clean Markdown
  • Structured URL discovery via the map command
  • Integration with Task agents to avoid context overload

Limitations

  • Requires a valid Firecrawl API key
  • Not a replacement for a search engine – limited to targeted pages
  • The quality of Markdown depends on the source page structure
When to use it

To fetch the textual content of a web page or explore the sitemap of a specific site.

When not to use it

For general web searches – prefer a search engine like Perplexity.

Security analysis

Safe
Quality score85/100

The skill instructs using the Firecrawl CLI for web scraping, a legitimate tool. No destructive commands, exfiltration, or obfuscation. It recommends using a Task agent to avoid context overload, indicating safe usage patterns. No security risks detected in the instruction file.

No concerns found

How this analysis is made

Examples

Scrape a page to Markdown
Scrape https://example.com/article to Markdown and summarize the key points.
Discover URLs on a site
Use Firecrawl to discover all pages under https://docs.example.com and list their URLs.

name: Firecrawl description: Web scraping and URL discovery using Firecrawl. Use for scraping web pages to markdown and discovering URLs on websites. Keywords: scrape, crawl, firecrawl, web, markdown, url discovery, sitemap.

Firecrawl

Web scraping and URL discovery powered by Firecrawl API.

Variables

  • FIRECRAWL_CLI_PATH: .claude/skills/firecrawl/firecrawl_cli/

Instructions

Run from FIRECRAWL_CLI_PATH:

cd .claude/skills/firecrawl/firecrawl_cli/
uv run fc --help                   # Discover all commands
uv run fc <command> --help         # Detailed usage

Rules:

  • ALWAYS use Task agent for scrapes - pages are large and will overload context
  • Use map to explore - when you need to find where content lives on a site
  • Use Perplexity for search - this skill is for targeted scraping, not search

Troubleshooting

  • "FIRECRAWL_API_KEY not found": Run /prime to validate environment

From the file to a first result

The download contains exactly the content displayed on this page. This copy may differ from the latest version in the source repository.

View the source and supporting files

This download contains SKILL.md only. Any scripts, references, templates and dependencies are not included. Get the required files from the source, preserving their directory structure, before continuing.

1. Download the file

Download SKILL.md

From the root of a test project, save the file as .skill-downloads/firecrawl-web-scraping-and-url-discovery-mmio71ef/SKILL.md, or use the command below. This folder is for inspecting the skill before installation.

Command for macOS/Linux or WSL: requires a POSIX shell and curl. On Windows without WSL, use the download button and create the folder manually.

mkdir -p '.skill-downloads/firecrawl-web-scraping-and-url-discovery-mmio71ef' && curl --fail --show-error --location 'https://skills-guides.com/api/skills/firecrawl-web-scraping-and-url-discovery-mmio71ef/download' --output '.skill-downloads/firecrawl-web-scraping-and-url-discovery-mmio71ef/SKILL.md.part' && mv '.skill-downloads/firecrawl-web-scraping-and-url-discovery-mmio71ef/SKILL.md.part' '.skill-downloads/firecrawl-web-scraping-and-url-discovery-mmio71ef/SKILL.md'

2. Check prerequisites

Read SKILL.md before using it. If it references scripts or external services, inspect them and prepare the required access in your test environment.

No recognized prerequisites section in this file. This does not mean it is self-contained: libraries, tools and access requirements may be mentioned elsewhere in the content or source repository.

The platforms on this page are catalog indications. The paths below follow tool documentation; they do not prove this skill has been run in those tools. No minimum tool version is confirmed for this listing.

3. Load the skill

This file does not provide a usable name/description header for this native procedure. It remains available to download and inspect; use the manual method below.

Manual method or another tool: attach SKILL.md to a conversation, or open it in an agent that can read files. Include any required supporting resources. No universal folder is recognized by every tool.

4. Try a first task

Use a short, synthetic input: an API endpoint, a small document or an interface brief. In a new conversation, attach the file or give the agent its path, then adapt this request:

Read .skill-downloads/firecrawl-web-scraping-and-url-discovery-mmio71ef/SKILL.md and apply its instructions to this example: [describe a test input]. Identify any missing files, dependencies or access. Produce the result requested by the skill without inventing data or running unreviewed scripts.

Expected result: output matching the skill instructions, or a precise list of missing prerequisites. A read confirmation alone does not validate a first use. If an external resource is unavailable, that path remains untested.

Related skills