---

name: firecrawl-data-handling description: | Implement FireCrawl PII handling, data retention, and GDPR/CCPA compliance patterns. Use when handling sensitive data, implementing data redaction, configuring retention policies, or ensuring compliance with privacy regulations for FireCrawl integrations. Trigger with phrases like "firecrawl data", "firecrawl PII", "firecrawl GDPR", "firecrawl data retention", "firecrawl privacy", "firecrawl CCPA". allowed-tools: Read, Write, Edit version: 1.0.0 license: MIT author: Jeremy Longshore jeremy@intentsolutions.io

FireCrawl Data Handling

Overview

Handle sensitive data correctly when integrating with FireCrawl.

Prerequisites

• Understanding of GDPR/CCPA requirements
• FireCrawl SDK with data export capabilities
• Database for audit logging
• Scheduled job infrastructure for cleanup

Data Classification

| Category | Examples | Handling | |----------|----------|----------| | PII | Email, name, phone | Encrypt, minimize | | Sensitive | API keys, tokens | Never log, rotate | | Business | Usage metrics | Aggregate when possible | | Public | Product names | Standard handling |

PII Detection

const PII_PATTERNS = [
  { type: 'email', regex: /[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/g },
  { type: 'phone', regex: /\b\d{3}[-.]?\d{3}[-.]?\d{4}\b/g },
  { type: 'ssn', regex: /\b\d{3}-\d{2}-\d{4}\b/g },
  { type: 'credit_card', regex: /\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b/g },
];

function detectPII(text: string): { type: string; match: string }[] {
  const findings: { type: string; match: string }[] = [];

  for (const pattern of PII_PATTERNS) {
    const matches = text.matchAll(pattern.regex);
    for (const match of matches) {
      findings.push({ type: pattern.type, match: match[0] });
    }
  }

  return findings;
}

Data Redaction

function redactPII(data: Record<string, any>): Record<string, any> {
  const sensitiveFields = ['email', 'phone', 'ssn', 'password', 'apiKey'];
  const redacted = { ...data };

  for (const field of sensitiveFields) {
    if (redacted[field]) {
      redacted[field] = '[REDACTED]';
    }
  }

  return redacted;
}

// Use in logging
console.log('FireCrawl request:', redactPII(requestData));

Data Retention Policy

Retention Periods

| Data Type | Retention | Reason | |-----------|-----------|--------| | API logs | 30 days | Debugging | | Error logs | 90 days | Root cause analysis | | Audit logs | 7 years | Compliance | | PII | Until deletion request | GDPR/CCPA |

Automatic Cleanup

async function cleanupFireCrawlData(retentionDays: number): Promise<void> {
  const cutoff = new Date();
  cutoff.setDate(cutoff.getDate() - retentionDays);

  await db.firecrawlLogs.deleteMany({
    createdAt: { $lt: cutoff },
    type: { $nin: ['audit', 'compliance'] },
  });
}

// Schedule daily cleanup
cron.schedule('0 3 * * *', () => cleanupFireCrawlData(30));

GDPR/CCPA Compliance

Data Subject Access Request (DSAR)

async function exportUserData(userId: string): Promise<DataExport> {
  const firecrawlData = await firecrawlClient.getUserData(userId);

  return {
    source: 'FireCrawl',
    exportedAt: new Date().toISOString(),
    data: {
      profile: firecrawlData.profile,
      activities: firecrawlData.activities,
      // Include all user-related data
    },
  };
}

Right to Deletion

async function deleteUserData(userId: string): Promise<DeletionResult> {
  // 1. Delete from FireCrawl
  await firecrawlClient.deleteUser(userId);

  // 2. Delete local copies
  await db.firecrawlUserCache.deleteMany({ userId });

  // 3. Audit log (required to keep)
  await auditLog.record({
    action: 'GDPR_DELETION',
    userId,
    service: 'firecrawl',
    timestamp: new Date(),
  });

  return { success: true, deletedAt: new Date() };
}

Data Minimization

// Only request needed fields
const user = await firecrawlClient.getUser(userId, {
  fields: ['id', 'name'], // Not email, phone, address
});

// Don't store unnecessary data
const cacheData = {
  id: user.id,
  name: user.name,
  // Omit sensitive fields
};

Instructions

Step 1: Classify Data

Categorize all FireCrawl data by sensitivity level.

Step 2: Implement PII Detection

Add regex patterns to detect sensitive data in logs.

Step 3: Configure Redaction

Apply redaction to sensitive fields before logging.

Step 4: Set Up Retention

Configure automatic cleanup with appropriate retention periods.

Output

• Data classification documented
• PII detection implemented
• Redaction in logging active
• Retention policy enforced

Error Handling

| Issue | Cause | Solution | |-------|-------|----------| | PII in logs | Missing redaction | Wrap logging with redact | | Deletion failed | Data locked | Check dependencies | | Export incomplete | Timeout | Increase batch size | | Audit gap | Missing entries | Review log pipeline |

Examples

Quick PII Scan

const findings = detectPII(JSON.stringify(userData));
if (findings.length > 0) {
  console.warn(`PII detected: ${findings.map(f => f.type).join(', ')}`);
}

Redact Before Logging

const safeData = redactPII(apiResponse);
logger.info('FireCrawl response:', safeData);

GDPR Data Export

const userExport = await exportUserData('user-123');
await sendToUser(userExport);

Resources

• GDPR Developer Guide
• CCPA Compliance Guide
• FireCrawl Privacy Guide

Next Steps

For enterprise access control, see firecrawl-enterprise-rbac.