Extraction API: extract(), bare_extraction(), and More

Trafilatura’s extraction module provides a suite of functions for pulling clean text and metadata from HTML documents. Whether you need a simple string result, a structured Python object, or just the raw text, these functions cover every use case — from quick one-liners to fully configured pipelines.

`extract()`

The primary entry point for text extraction. Takes an HTML document in any supported form and returns the extracted content as a formatted string, or None if extraction fails or the document doesn’t meet quality thresholds.

from trafilatura import extract, fetch_url

html = fetch_url("https://example.com/article")
result = extract(html)
print(result)

Signature

extract(
    filecontent,
    url=None,
    record_id=None,
    fast=False,
    no_fallback=False,
    favor_precision=False,
    favor_recall=False,
    include_comments=True,
    output_format="txt",
    tei_validation=False,
    target_language=None,
    include_tables=True,
    include_images=False,
    include_formatting=None,
    include_links=False,
    deduplicate=False,
    date_extraction_params=None,
    with_metadata=False,
    only_with_metadata=False,
    max_tree_size=None,
    url_blacklist=None,
    author_blacklist=None,
    settingsfile=None,
    prune_xpath=None,
    config=DEFAULT_CONFIG,
    options=None,
) -> str | None

Parameters

filecontent

str | bytes | HtmlElement

required

The HTML input to extract from. Accepts a Unicode string, raw bytes, or an already-parsed lxml.html.HtmlElement tree.

url

str

default:"None"

URL of the webpage. Used for metadata extraction and resolving relative links.

record_id

str

default:"None"

An optional identifier added to the document metadata (available in structured output formats like XML, JSON, and CSV).

fast

bool

default:"False"

Skip the fallback extraction algorithms (readability and jusText) and use only Trafilatura’s main extractor. Faster but may produce shorter or lower-quality output for difficult pages.

no_fallback

bool

default:"False"

Deprecated. Use fast=True instead. Raises a DeprecationWarning.

favor_precision

bool

default:"False"

Prefer shorter, higher-confidence extractions. Reduces false positives at the cost of possibly missing some content.

favor_recall

bool

default:"False"

Prefer more text even when uncertain. Useful when missing content is worse than including some noise.

include_comments

bool

default:"True"

Extract user comments from comment sections along with the main article body.

output_format

str

default:"\"txt\""

The serialization format for the returned string. One of:

"txt" — plain text (default)
"markdown" — Markdown with heading and emphasis preservation
"html" — cleaned HTML
"json" — JSON object
"csv" — tab-separated values
"xml" — Trafilatura XML format
"xmltei" — TEI-compliant XML

tei_validation

bool

default:"False"

Validate the output against the TEI standard. Only applicable when output_format="xmltei".

target_language

str

default:"None"

Discard documents that are not written in the specified language. Accepts an ISO 639-1 two-letter language code (e.g., "en", "de"). Requires py3langid to be installed for full functionality.

include_tables

bool

default:"True"

Extract text from HTML <table> elements.

include_images

bool

default:"False"

Include image elements in the extraction (experimental). Captures src and alt attributes.

include_formatting

bool

default:"None"

Preserve structural formatting elements such as bold, italic, and headings. Rendered as Markdown in text formats; kept as tags in XML formats. Ignored for JSON output.

include_links

bool

default:"False"

Keep hyperlinks and their targets in the output (experimental).

deduplicate

bool

default:"False"

Filter out duplicate text segments and previously seen documents using a fingerprint cache.

date_extraction_params

dict

default:"None"

A dictionary of keyword arguments forwarded to htmldate for date extraction. For example: {"original_date": True, "extensive_search": False}.

with_metadata

bool

default:"False"

Prepend extracted metadata (title, author, date, etc.) to the output string. For "txt" and "markdown" formats this is rendered as a YAML front matter block.

only_with_metadata

bool

default:"False"

Return None and discard documents that are missing any of the three essential metadata fields: date, title, or url.

max_tree_size

int

default:"None"

Deprecated. Raises a ValueError when set. Configure tree size limits via the settings file instead.

url_blacklist

set[str]

default:"None"

A set of URLs to exclude. Documents whose resolved URL appears in this set are discarded and None is returned.

author_blacklist

set[str]

default:"None"

A set of author name strings to suppress. Matching author values are removed from the extracted metadata.

settingsfile

str

default:"None"

Path to a .cfg configuration file that overrides the default Trafilatura settings (timeouts, size limits, etc.).

prune_xpath

str | list[str]

default:"None"

One or more XPath expressions. Matching elements are removed from the parse tree before extraction begins. Useful for stripping site-specific boilerplate.

config

ConfigParser

default:"DEFAULT_CONFIG"

A configparser.ConfigParser instance with Trafilatura settings. Superseded by options when both are provided.

options

Extractor

default:"None"

A pre-built trafilatura.settings.Extractor object. When provided, all individual keyword arguments are ignored in favour of the options object.

Returns: str | None — The extracted content in the chosen format, or None if extraction fails. Examples

from trafilatura import extract, fetch_url

# Basic extraction
html = fetch_url("https://www.example.com/article")
text = extract(html)

# JSON output with metadata
result = extract(
    html,
    url="https://www.example.com/article",
    output_format="json",
    with_metadata=True,
    include_formatting=True,
)

# Precision mode, English only, skip duplicates
text = extract(
    html,
    favor_precision=True,
    target_language="en",
    deduplicate=True,
)

# Remove sidebar and cookie banners before extracting
text = extract(
    html,
    prune_xpath=["//aside", "//div[@id='cookie-banner']"],
)

# Markdown with links and tables
markdown = extract(
    html,
    output_format="markdown",
    include_formatting=True,
    include_links=True,
    include_tables=True,
)

`bare_extraction()`

A lower-level extraction function that returns a Document object instead of a serialized string. The Document.body attribute is a live lxml element tree, allowing further programmatic manipulation before serialization.

from trafilatura import bare_extraction, fetch_url

html = fetch_url("https://example.com/article")
doc = bare_extraction(html, url="https://example.com/article", with_metadata=True)
if doc:
    print(doc.title)
    print(doc.text)

Signature

bare_extraction(
    filecontent,
    url=None,
    fast=False,
    no_fallback=False,
    favor_precision=False,
    favor_recall=False,
    include_comments=True,
    output_format="python",
    target_language=None,
    include_tables=True,
    include_images=False,
    include_formatting=None,
    include_links=False,
    deduplicate=False,
    date_extraction_params=None,
    with_metadata=False,
    only_with_metadata=False,
    max_tree_size=None,
    url_blacklist=None,
    author_blacklist=None,
    as_dict=False,
    prune_xpath=None,
    config=DEFAULT_CONFIG,
    options=None,
) -> Document | dict | None

Parameters

filecontent

any

required

HTML content as a Unicode string, bytes, or a pre-parsed lxml.html.HtmlElement.

url

str

default:"None"

URL of the source page, used for metadata extraction and relative link resolution.

fast

bool

default:"False"

Skip fallback extractors (readability, jusText). Faster with potentially less complete output.

no_fallback

bool

default:"False"

Deprecated. Use fast=True instead.

favor_precision

bool

default:"False"

Prefer shorter, higher-confidence results.

favor_recall

bool

default:"False"

Prefer more text, even if uncertain.

include_comments

bool

default:"True"

Include comment sections in the extraction.

output_format

str

default:"\"python\""

Serialization format. Defaults to "python" which leaves body as a raw lxml element. Other valid values are the same as extract().

target_language

str

default:"None"

ISO 639-1 language code. Discards documents in other languages.

include_tables

bool

default:"True"

Include <table> content in extraction.

include_images

bool

default:"False"

Include image metadata (experimental).

include_formatting

bool

default:"None"

Preserve formatting tags (bold, italic, headings).

include_links

bool

default:"False"

Preserve hyperlinks (experimental).

deduplicate

bool

default:"False"

Filter duplicates using a content fingerprint.

date_extraction_params

dict

default:"None"

Parameters forwarded to htmldate for date extraction.

with_metadata

bool

default:"False"

Extract and populate metadata fields on the Document object.

only_with_metadata

bool

default:"False"

Return None when date, title, or url are missing.

max_tree_size

int

default:"None"

Deprecated. Raises a ValueError when set. Configure tree size limits via the settings file instead.

url_blacklist

set[str]

default:"None"

Set of URLs to exclude from results.

author_blacklist

set[str]

default:"None"

Set of author names to suppress.

as_dict

bool

default:"False"

Deprecated. Use document.as_dict() instead.

prune_xpath

str | list[str]

default:"None"

XPath expression(s) for elements to remove before extraction.

config

ConfigParser

default:"DEFAULT_CONFIG"

Configuration object for Trafilatura settings.

options

Extractor

default:"None"

Pre-built Extractor options object. Overrides all individual parameters.

Returns: Document | None — a Document object, or None on failure. The Document object

Attribute	Type	Description
`title`	`str \| None`	Page title
`author`	`str \| None`	Author name(s)
`url`	`str \| None`	Canonical URL
`hostname`	`str \| None`	Domain name
`description`	`str \| None`	Meta description
`sitename`	`str \| None`	Publisher / site name
`date`	`str \| None`	Publication date (ISO format)
`categories`	`list[str] \| None`	Article categories
`tags`	`list[str] \| None`	Article tags or keywords
`fingerprint`	`str \| None`	Content fingerprint hash
`id`	`str \| None`	Record identifier
`license`	`str \| None`	Content license (e.g. CC-BY)
`language`	`str \| None`	Detected language code
`image`	`str \| None`	Featured image URL
`pagetype`	`str \| None`	Page type classification
`body`	`lxml._Element`	Extracted content as an lxml element tree
`comments`	`str \| None`	Extracted comments as plain text
`commentsbody`	`lxml._Element`	Extracted comments as an lxml element tree
`text`	`str \| None`	Serialized text content

Call document.as_dict() to convert the object to a plain Python dictionary. Examples

from trafilatura import bare_extraction, fetch_url

html = fetch_url("https://www.example.com/article")

# Get a Document object with metadata
doc = bare_extraction(
    html,
    url="https://www.example.com/article",
    with_metadata=True,
    include_formatting=True,
)

if doc:
    print(f"Title: {doc.title}")
    print(f"Author: {doc.author}")
    print(f"Date: {doc.date}")
    print(f"Text preview: {doc.text[:200]}")

    # Convert to dict for serialization
    data = doc.as_dict()
    print(data["title"])

# Filter by language and avoid duplicates
doc = bare_extraction(
    html,
    target_language="de",
    deduplicate=True,
    only_with_metadata=True,
)

`extract_with_metadata()`

A convenience wrapper around extract() that always extracts metadata and returns a Document object instead of a plain string. Equivalent to calling extract() with with_metadata=True but returning the full Document rather than document.text.

from trafilatura import extract_with_metadata, fetch_url

html = fetch_url("https://example.com/article")
doc = extract_with_metadata(html, url="https://example.com/article")
if doc:
    print(doc.title, doc.date, doc.text)

Signature

extract_with_metadata(
    filecontent,
    url=None,
    record_id=None,
    fast=False,
    favor_precision=False,
    favor_recall=False,
    include_comments=True,
    output_format="txt",
    tei_validation=False,
    target_language=None,
    include_tables=True,
    include_images=False,
    include_formatting=None,
    include_links=False,
    deduplicate=False,
    date_extraction_params=None,
    url_blacklist=None,
    author_blacklist=None,
    settingsfile=None,
    prune_xpath=None,
    config=DEFAULT_CONFIG,
    options=None,
) -> Document | None

Parameters

filecontent

str | bytes | HtmlElement

required

HTML input as a string, bytes, or a pre-parsed lxml tree.

url

str

default:"None"

Source URL for metadata extraction and link resolution.

record_id

str

default:"None"

Optional record identifier written to document.id.

fast

bool

default:"False"

Skip fallback extractors.

favor_precision

bool

default:"False"

Prefer fewer, higher-confidence results.

favor_recall

bool

default:"False"

Prefer more text, even when uncertain.

include_comments

bool

default:"True"

Include comment sections in the output.

output_format

str

default:"\"txt\""

Format for document.text. Same options as extract().

tei_validation

bool

default:"False"

Validate TEI XML output (only relevant when output_format="xmltei").

target_language

str

default:"None"

ISO 639-1 code. Documents in other languages are discarded.

include_tables

bool

default:"True"

Extract table content.

include_images

bool

default:"False"

Include image metadata (experimental).

include_formatting

bool

default:"None"

Preserve formatting elements.

include_links

bool

default:"False"

Preserve hyperlinks (experimental).

deduplicate

bool

default:"False"

Filter duplicate documents.

date_extraction_params

dict

default:"None"

Parameters for htmldate date extraction.

url_blacklist

set[str]

default:"None"

URLs to exclude.

author_blacklist

set[str]

default:"None"

Author names to suppress.

settingsfile

str

default:"None"

Path to a Trafilatura settings .cfg file.

prune_xpath

str | list[str]

default:"None"

XPath expressions for elements to prune before extraction.

config

ConfigParser

default:"DEFAULT_CONFIG"

Configuration object.

options

Extractor

default:"None"

Pre-built extractor options. Overrides all individual parameters.

Returns: Document | None Example

from trafilatura import extract_with_metadata, fetch_url

urls = [
    "https://www.example.com/article-1",
    "https://www.example.com/article-2",
]

for url in urls:
    html = fetch_url(url)
    doc = extract_with_metadata(
        html,
        url=url,
        output_format="markdown",
        include_formatting=True,
        target_language="en",
    )
    if doc:
        print(f"[{doc.date}] {doc.title} by {doc.author}")
        print(doc.text[:300])

`extract_metadata()`

Extracts only the metadata of a web page — title, author, date, URL, description, etc. — without performing full text extraction. Useful when you need structured metadata without the overhead of content analysis.

from trafilatura import extract_metadata, fetch_url

html = fetch_url("https://example.com/article")
meta = extract_metadata(html, default_url="https://example.com/article")
print(meta.title, meta.author, meta.date)

Signature

extract_metadata(
    filecontent,
    default_url=None,
    date_config=None,
    extensive=True,
    author_blacklist=None,
) -> Document

Parameters

filecontent

str | HtmlElement

required

The HTML document as a string or a pre-parsed lxml.html.HtmlElement tree.

default_url

str

default:"None"

A previously known URL for the document. Used as a fallback when the canonical URL cannot be extracted from the HTML itself.

date_config

dict

default:"None"

A dictionary of parameters forwarded to htmldate for publication date extraction. If None, defaults are applied with extensive=True.

extensive

bool

default:"True"

Enable extensive date search, which tries more extraction strategies at the cost of some performance.

author_blacklist

set[str]

default:"None"

A set of author name strings to exclude from the metadata output.

Returns: Document — always returns a Document instance (never None). Fields will be None when the corresponding metadata cannot be found. Example

from trafilatura import extract_metadata, fetch_url

html = fetch_url("https://www.theguardian.com/some-article")
meta = extract_metadata(
    html,
    default_url="https://www.theguardian.com/some-article",
    extensive=False,  # faster, less thorough date search
)

print(f"Title:       {meta.title}")
print(f"Author:      {meta.author}")
print(f"Date:        {meta.date}")
print(f"Description: {meta.description}")
print(f"Sitename:    {meta.sitename}")
print(f"License:     {meta.license}")

# Serialize to dict
data = meta.as_dict()

`html2txt()`

A simple utility that strips all HTML tags and returns the raw text content of a document. Unlike extract(), this function performs no content analysis or quality filtering — it returns everything.

from trafilatura import html2txt

raw_html = "<html><body><h1>Hello</h1><p>World</p></body></html>"
text = html2txt(raw_html)
# → "Hello World"

Signature

html2txt(content, clean=True) -> str

Parameters

content

str | HtmlElement

required

The HTML input as a Unicode string or a pre-parsed lxml.html.HtmlElement tree.

clean

bool

default:"True"

Remove potentially undesirable elements (scripts, styles, navigation, etc.) before text extraction. Set to False to keep all content including boilerplate.

Returns: str — the extracted text as a single string, with whitespace normalized. Returns an empty string if parsing fails. Examples

from trafilatura import html2txt, fetch_url

# From a string
html = "<article><h1>Breaking News</h1><p>Details here.</p></article>"
print(html2txt(html))
# → "Breaking News Details here."

# From a downloaded page
raw = fetch_url("https://example.com")
if raw:
    print(html2txt(raw))

# Disable cleaning to keep all text (including nav, footer, etc.)
full_text = html2txt(raw, clean=False)

`baseline()`

A minimal, dependency-free extraction function that uses simple heuristics to find the main text. It tries (in order): JSON-LD articleBody, <article> elements, paragraph tags, and finally the full body text. Primarily used as an internal fallback within the main extraction pipeline.

from trafilatura.baseline import baseline

html = "<html><body><article><p>Main content here.</p></article></body></html>"
body_element, text, length = baseline(html)
print(text)   # → "Main content here."
print(length) # → 18

Signature

baseline(filecontent) -> tuple[lxml._Element, str, int]

Parameters

filecontent

str | bytes | HtmlElement

required

HTML code as a binary string, Unicode string, or parsed lxml.html.HtmlElement.

Returns: A 3-tuple of:

lxml._Element — a <body> element containing the extracted paragraphs as <p> children
str — the main text content as a plain string
int — the character length of the extracted text

Examples

from trafilatura.baseline import baseline
from trafilatura import fetch_url

html = fetch_url("https://example.com/article")

if html:
    body_elem, text, length = baseline(html)

    print(f"Extracted {length} characters")
    print(text[:500])

    # The element tree can be further inspected
    for para in body_elem.findall(".//p"):
        print("-", para.text)

Core Functions

Configuration

Extraction API: extract(), bare_extraction(), and More

`extract()`

`bare_extraction()`

`extract_with_metadata()`

`extract_metadata()`

`html2txt()`

`baseline()`

Build docs developers (and LLMs) love

Core Functions

Configuration

Documentation Index

​extract()

​bare_extraction()

​extract_with_metadata()

​extract_metadata()

​html2txt()

​baseline()

Build docs developers (and LLMs) love

`extract()`

`bare_extraction()`

`extract_with_metadata()`

`extract_metadata()`

`html2txt()`

`baseline()`