Documentation Index
Fetch the complete documentation index at: https://mintlify.com/AgricIDaniel/claude-seo/llms.txt
Use this file to discover all available pages before exploring further.
The Firecrawl extension adds full-site crawling, URL discovery, and JavaScript-rendered scraping to Claude SEO via the Firecrawl MCP server. Where the core skill fetches individual pages, Firecrawl maps and crawls entire sites in a single pass — including React, Vue, Next.js, and other JavaScript-heavy sites that require a browser to render their content. This is especially useful for /seo audit on larger or more complex sites where the core skill might miss pages behind client-side routing or lazy-loaded link structures.
What Firecrawl adds
- Full-site crawl — extract content from every page on a domain, up to 500 pages per crawl
- Site mapping — discover all URLs without fetching content (fast and credit-efficient)
- JS-rendered scraping — render JavaScript before extraction, solving the SPA content problem
- Site-scoped search — search within a site for specific content without crawling everything
The free tier provides 500 credits per month. One credit equals one page crawled or scraped; map operations cost 0.5 credits per URL discovered.
Prerequisites
Installation
Clone the repo (if you haven't already)
git clone --depth 1 https://github.com/AgriciDaniel/claude-seo.git
cd claude-seo
Run the installer
macOS / Linux
Windows PowerShell
./extensions/firecrawl/install.sh
.\extensions\firecrawl\install.ps1
~/.claude/settings.json automatically.Restart Claude Code
Close and reopen Claude Code so it picks up the new MCP server entry.
Verify
/seo firecrawl map https://example.com
Commands
| Command | Purpose | Credits |
|---|
/seo firecrawl crawl <url> | Full-site crawl with content extraction | 1 per page |
/seo firecrawl map <url> | Discover site structure (URLs only, no content) | 0.5 per URL |
/seo firecrawl scrape <url> | Single-page deep scrape with JS rendering | 1 |
/seo firecrawl search <query> <url> | Search within a site for specific content | 1 per result |
crawl — full-site crawl
Crawls an entire website from the given URL, extracting page content, metadata, and links for all discovered pages. Supports includePaths and excludePaths glob patterns to focus the crawl on a section (for example, /blog/* only) and an adjustable depth limit (default: 3 levels, max: 500 pages).
# Crawl the full site
/seo firecrawl crawl https://example.com
# For very large sites, the skill caps at 100 pages by default
# and prompts before going further to prevent unexpected credit usage
map — URL discovery
The fastest way to get a complete URL list without fetching any page content. Use this first on large sites, then pipe the results into a targeted crawl of the most important sections.
/seo firecrawl map https://example.com
/blog/*, /products/*, /category/*, and so on — so you can spot URL structure problems and crawl budget waste at a glance.
scrape — single-page JS rendering
Renders a single page with full JavaScript execution before extraction. More thorough than the core skill’s fetch_page.py for SPA-based sites, and capable of waiting for dynamic content below the fold.
/seo firecrawl scrape https://example.com/blog/post
For static HTML pages, the core skill’s fetch_page.py is faster and costs no credits. Use scrape when a page is JS-rendered, returns an empty <div id="root"> shell, or requires interaction before content appears.
search — site-scoped content search
Searches within a website for pages matching a query without crawling the whole site. Useful for checking whether content on a specific topic already exists, finding internal linking opportunities, or scanning a competitor site for a subject area.
/seo firecrawl search "keyword research" https://example.com
Credit pricing
| Plan | Credits/month | Price |
|---|
| Free | 500 | $0 |
| Hobby | 3,000 | $16/mo |
| Standard | 100,000 | $83/mo |
| Growth | 500,000 | $333/mo |
How it complements the core audit
When Firecrawl is active during /seo audit, the orchestrator follows this pattern:
map — discover all site URLs quickly (0.5 credits/URL)- Compare discovered URLs against the XML sitemap to surface orphan pages and stale sitemap entries
crawl on the 50 most important pages (homepage, key sections, top-traffic URLs)- Feed crawled content to all subagents — technical, content, schema, geo — for analysis at scale
Without Firecrawl, the core skill analyzes pages you point it at explicitly. With Firecrawl, it can audit a 300-page site holistically in a single /seo audit run.
Integration with core skills
| Skill | How Firecrawl helps |
|---|
/seo audit | Uses map for full URL discovery, crawl for deep multi-page analysis |
/seo technical | Broken link detection across the entire site; redirect chain mapping |
/seo sitemap | Compares XML sitemap vs. actually crawlable pages; flags orphan pages and 404 entries |
/seo content | Thin content detection at scale; near-duplicate identification across pages |
/seo schema | Schema extraction and coverage reporting across all crawled pages |
Uninstall
./extensions/firecrawl/uninstall.sh # macOS / Linux
.\extensions\firecrawl\uninstall.ps1 # Windows
Troubleshooting
| Symptom | Cause | Fix |
|---|
| MCP not connecting | API key not in settings | Run cat ~/.claude/settings.json | python3 -m json.tool | grep firecrawl to check; re-run installer if missing |
402 Payment Required | Credits exhausted | Check usage at firecrawl.dev/app; upgrade or wait for monthly reset |
403 Forbidden | Site blocks crawling via robots.txt or Cloudflare | Try scrape (single page) instead of crawl; some sites require manual fetching |
408 Timeout | Page renders too slowly | Increase the timeout parameter; try without JS rendering for static pages |
