Documentation Index
Fetch the complete documentation index at: https://mintlify.com/Amaculus/screaming-frog-api/llms.txt
Use this file to discover all available pages before exploring further.
In addition to the typed crawl.pages() and crawl.links() views, you can access any exported tab by name using crawl.tab().
Accessing a tab
from screamingfrog import Crawl
crawl = Crawl.load("./exports")
# Access a tab by file name (extension optional)
for row in crawl.tab("response_codes_all"):
print(row["Address"], row["Status Code"])
"Page Titles" → resolves to page_titles_all.csv"page_titles" → resolves to page_titles_all.csv"page_titles_all" → used directly
Listing available tabs
Use crawl.tabs to see which tabs are available for the current backend:
CSV backend — exposes any *.csv file found in the export directory.Derby backend — exposes all tabs mapped in schemas/mapping.json (601 of 628 tabs fully mapped).SQLite backend — supports a smaller high-value set: response codes, page titles, meta description, and internal_all.
Filtering tab rows
Filter using column names (or their snake_case equivalents):
for row in crawl.tab("internal_all").filter(status_code="404"):
print(row["Address"])
GUI filter shortcut
Apply a Screaming Frog GUI filter by name using the gui keyword:
for row in crawl.tab("page_titles").filter(gui="Missing"):
print(row["Address"], row["Title 1"])
for row in crawl.tab("page_titles").filter(gui_filters=["Missing", "Duplicate"]):
print(row["Address"])
For exact GUI filter behaviour, use CSV exports. The export_profile="kitchen_sink" option exports all filter variants from the Screaming Frog UI.
List GUI filter names
print(crawl.tab_filters("Page Titles"))
# ['Missing', 'Duplicate', 'Over 60 Characters', 'Below 30 Characters', ...]
Inspect columns
print(crawl.tab_columns("page_titles"))
# ['Address', 'Title 1', 'Title 1 Length', 'Title 1 Pixel Width', ...]
Describe a tab (columns + filters in one call)
print(crawl.describe_tab("page_titles"))
# {
# 'tab': 'page_titles',
# 'columns': ['Address', 'Title 1', ...],
# 'filters': ['Missing', 'Duplicate', ...]
# }
TabView methods
crawl.tab() returns a TabView. All standard view methods are available:
tab = crawl.tab("response_codes_all")
# Count rows
count = tab.filter(status_code="404").count()
# Collect as list
rows = tab.filter(status_code="404").collect()
# First row
first = tab.first()
# Convert to DataFrame
df = tab.to_pandas()
df_polars = tab.to_polars()
Backend differences
CSV
Derby
Hybrid Derby+CSV
Exposes any *.csv file in the export directory. GUI filters rely on exported filter variants — use export_profile="kitchen_sink" to get all filter columns.crawl = Crawl.load("./exports")
for row in crawl.tab("page_titles").filter(gui="Missing"):
print(row["Address"])
Exposes all 601 mapped tabs from schemas/mapping.json. Derby natively supports tabs that require computed fields:
Response Codes > Internal Redirect ChainHreflang > Not Using Canonical
Derby-backed crawl.internal also materialises computed mapped fields like Indexability and Indexability Status.Derby filters work against mapped expression fields and header-derived fields in both crawl.internal and crawl.tab(...).crawl = Crawl.load("./crawl.dbseospider", dbseospider_backend="derby", csv_fallback=False)
for row in crawl.tab("hreflang_not_using_canonical"):
print(row["Address"])
The default for Crawl.load with Derby sources. Exports missing tabs on demand and caches them next to the crawl.# Default — CSV fallback enabled
crawl = Crawl.load("./crawl.dbseospider")
csv_fallback=False to disable automatic CSV exports:crawl = Crawl.load("./crawl.dbseospider", csv_fallback=False)
Full example: response codes
from screamingfrog import Crawl
crawl = Crawl.load("./exports")
# List available tabs
print(crawl.tabs)
# All 4xx and 5xx pages
for row in crawl.tab("response_codes_all").filter(status_code="404"):
print(row["Address"], row["Status Code"])
# Check available filters
print(crawl.tab_filters("Response Codes"))
# Apply a GUI filter by name
for row in crawl.tab("response_codes_all").filter(gui="Internal Client Error (4xx)"):
print(row["Address"])