Skills Guide

Web Scraping with Sitepanda

VerifiedCaution

Scrape websites with a headless browser and extract main readable content as Markdown.

Download and use
Sby Skills Guide Bot
Data & AIBeginner
7/28/2026
Claude CodeCursorWindsurfCodex
#web-scraping#markdown-extraction#headless-browser#content-summarization#url-fetching

Recommended for

Claude CodeCursorWindsurfCodexData scientistData & analysis

Automated summary

This skill lets the agent fetch a web page through the Sitepanda CLI and extract its main readable content as Markdown for analysis or summarization.

Strengths

  • Output is Markdown, so the agent can read and analyze it without extra parsing
  • One straightforward command (sitepanda scrape <URL> --silent --limit 1) that keeps scraping to a single page
  • Recursive scraping requires explicit user confirmation, preventing long or costly crawls
  • Includes an instruction to summarize or trim long pages so responses stay concise

Limitations

  • Only the CLI invocation is described; installation of Sitepanda and error handling for failed fetches are not covered
  • No guidance for JavaScript-heavy sites, paywalls, or rate limiting
  • Extraction quality depends entirely on the tool's main-content detection, with no custom selector option
When to use it

Use it whenever the user supplies a URL and wants the page's content retrieved, summarized, or explained.

When not to use it

Avoid it when the task requires crawling a whole site or scraping pages behind authentication, which this skill does not address.

Security analysis

Caution
Quality score78/100

The skill performs legitimate web scraping using a headless browser, but it involves network access and an external command. While not inherently destructive or exfiltrating, it can be abused to reach internal resources or fetch malicious content, so caution is warranted.

Findings
  • •Invokes an external network tool (`sitepanda`) to fetch and scrape URLs, which could be used to access internal or sensitive endpoints if an untrusted URL is supplied.
  • •No allowed-tools are declared despite relying on an external command, making execution scope unclear.
  • •Recursive scraping is gated by user confirmation, but single-page scraping of arbitrary URLs is not restricted.
How this analysis is made

Examples

Summarize an article
Summarize the article at https://example.com/blog/post-123
Explain a docs page
What does this documentation page say? https://example.com/docs
Extract key points
Scrape https://example.com/pricing and list the key points of the pricing page.

name: sitepanda description: > Scrape websites with a headless browser and extract main readable content as Markdown. Use this skill when the user asks to retrieve, analyze, or summarize content from a URL or website.

Sitepanda (Web Scraping Tool)

Instructions

  1. When the user provides a URL or asks for website content, use Sitepanda to scrape the page.

  2. By default, use the following command to scrape a single page:

    sitepanda scrape <URL> --silent --limit 1

  3. If you need to perform recursive scraping (following links), you must ask the user for confirmation before starting, as it may take a long time.

  4. Capture the output, which is returned in Markdown format.

  5. Read and analyze the extracted content.

  6. Respond to the user using only the relevant information from the page.

  7. If the content is long, summarize or extract only the necessary sections.

Examples

Example 1

User request: "Please summarize the article at https://example.com/blog/post-123"

Agent behavior:

  • Use Sitepanda to scrape the page
  • Read the extracted Markdown
  • Summarize the main points in the response

Example 2

User request: "What does this documentation page say? https://example.com/docs"

Agent behavior:

  • Fetch the page using Sitepanda
  • Extract key sections
  • Explain the content concisely

From the file to a first result

The download contains exactly the content displayed on this page. This copy may differ from the latest version in the source repository.

View the source and supporting files

This download contains SKILL.md only. Any scripts, references, templates and dependencies are not included. Get the required files from the source, preserving their directory structure, before continuing.

1. Download the file

Download SKILL.md

From the root of a test project, save the file as .skill-downloads/web-scraping-with-sitepanda-ms47vpjs/SKILL.md, or use the command below. This folder is for inspecting the skill before installation.

Command for macOS/Linux or WSL: requires a POSIX shell and curl. On Windows without WSL, use the download button and create the folder manually.

mkdir -p '.skill-downloads/web-scraping-with-sitepanda-ms47vpjs' && curl --fail --show-error --location 'https://skills-guides.com/api/skills/web-scraping-with-sitepanda-ms47vpjs/download' --output '.skill-downloads/web-scraping-with-sitepanda-ms47vpjs/SKILL.md.part' && mv '.skill-downloads/web-scraping-with-sitepanda-ms47vpjs/SKILL.md.part' '.skill-downloads/web-scraping-with-sitepanda-ms47vpjs/SKILL.md'

2. Check prerequisites

Read SKILL.md before using it. If it references scripts or external services, inspect them and prepare the required access in your test environment.

No recognized prerequisites section in this file. This does not mean it is self-contained: libraries, tools and access requirements may be mentioned elsewhere in the content or source repository.

The platforms on this page are catalog indications. The paths below follow tool documentation; they do not prove this skill has been run in those tools. No minimum tool version is confirmed for this listing.

3. Load the skill

After inspection, copy the complete folder into your project using one of the commands below, then open a new tool session. The command stops if a folder with the same name already exists: compare the files before replacing it.

Claude Code
test -f '.skill-downloads/web-scraping-with-sitepanda-ms47vpjs/SKILL.md' && test -s '.skill-downloads/web-scraping-with-sitepanda-ms47vpjs/SKILL.md' && mkdir -p '.claude/skills' && test ! -e '.claude/skills/sitepanda' && cp -R '.skill-downloads/web-scraping-with-sitepanda-ms47vpjs' '.claude/skills/sitepanda'

Cursor
test -f '.skill-downloads/web-scraping-with-sitepanda-ms47vpjs/SKILL.md' && test -s '.skill-downloads/web-scraping-with-sitepanda-ms47vpjs/SKILL.md' && mkdir -p '.cursor/skills' && test ! -e '.cursor/skills/sitepanda' && cp -R '.skill-downloads/web-scraping-with-sitepanda-ms47vpjs' '.cursor/skills/sitepanda'

Codex
test -f '.skill-downloads/web-scraping-with-sitepanda-ms47vpjs/SKILL.md' && test -s '.skill-downloads/web-scraping-with-sitepanda-ms47vpjs/SKILL.md' && mkdir -p '.agents/skills' && test ! -e '.agents/skills/sitepanda' && cp -R '.skill-downloads/web-scraping-with-sitepanda-ms47vpjs' '.agents/skills/sitepanda'

OpenSkills

OpenSkills must already be installed. The local install/read procedure is checked with OpenSkills 1.5.0, which requires Node.js ≥ 20.6.0. Catalog slugs are not resolved automatically.

Official OpenSkills installation
openskills install './.skill-downloads/web-scraping-with-sitepanda-ms47vpjs'

openskills read 'web-scraping-with-sitepanda-ms47vpjs'

Reading should display the installed file content and its base directory. It does not run the skill or install its dependencies.

For an agent that reads AGENTS.md, then sync the index. This command changes AGENTS.md: review the diff before using it.

openskills sync

Manual method or another tool: attach SKILL.md to a conversation, or open it in an agent that can read files. Include any required supporting resources. No universal folder is recognized by every tool.

4. Try a first task

Use a short, synthetic input: an API endpoint, a small document or an interface brief. In a new conversation, attach the file or give the agent its path, then adapt this request:

Read .skill-downloads/web-scraping-with-sitepanda-ms47vpjs/SKILL.md and apply its instructions to this example: [describe a test input]. Identify any missing files, dependencies or access. Produce the result requested by the skill without inventing data or running unreviewed scripts.

Expected result: output matching the skill instructions, or a precise list of missing prerequisites. A read confirmation alone does not validate a first use. If an external resource is unavailable, that path remains untested.

Related skills

Create Interactive Maps with kepler.gl

Data & AI

Create interactive map visualizations and export standalone HTML using the keplergl Python package. Use to plot geospatial data from DataFrames, GeoJSON, or CSV.

Claude CodeCursorCodexintermediateSafe 11,995
Skills Guide Bot

Agent-Driven Parameter Optimization

Data & AI

Guide for optimizing configurable systems with a measurable objective, using bounded trials and evidence-based recommendations.

Claude CodeCursorCodexintermediateSafe 45,609
Skills Guide Bot

ICP Visitor Outreach

Data & AI

Identify signed-up users matching the ideal customer profile but stalled, and draft personalized outreach to re-engage them.

Claude CodeintermediateSafe 6,843
Skills Guide Bot