Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/adbar/trafilatura/llms.txt

Use this file to discover all available pages before exploring further.

Trafilatura supports seven output formats that cover a range of use cases — from simple plain-text pipelines to structured archival formats. The default is plain text ("txt"). Select a format via the output_format parameter in Python or via a flag on the CLI.

TXT

Plain text. Default format. Optionally adds a YAML metadata header.

Markdown

Plain text with inline formatting (bold, italic, headings). Available from version 1.9.

JSON

Machine-readable JSON object. Includes all metadata fields when requested.

CSV

Tab-separated values. Fixed column order including URL, title, date, and body text.

HTML

Cleaned, minimal HTML. Available from version 1.11.

XML

Structured XML preserving document hierarchy, formatting, and links.

XML-TEI

TEI-compliant XML following the Text Encoding Initiative guidelines. Ideal for archival.

​
TXT (Plain Text)

The default output format. Extracts and returns the main text content as a clean, readable string. Comments are appended below the main text when present. What it includes: Main body text, optionally comments, and optionally a YAML front-matter metadata header (when with_metadata=True). Example output: 
This is the opening paragraph of the article.

Another paragraph follows here with additional detail.
With with_metadata=True: 
---
title: Example Article
author: Jane Doe
url: https://www.example.org/article
date: 2024-02-01
---
This is the opening paragraph of the article.

from trafilatura import extract, fetch_url

downloaded = fetch_url("https://www.example.org/article")

# plain text (default)
result = extract(downloaded)

# plain text with YAML metadata header
result = extract(downloaded, with_metadata=True)

# explicit format
result = extract(downloaded, output_format="txt")
Adding include_formatting=True or include_links=True to a TXT extraction automatically switches the output to Markdown format to accommodate the extra structural elements.

​
Markdown

Returns plain text with inline formatting preserved. Headings, bold, italic, and links are rendered as Markdown syntax. Available from version 1.9 onwards. Selecting Markdown automatically enables formatting — you do not need to pass include_formatting=True separately. What it includes: Main body text with Markdown-rendered formatting, headings, and (optionally) links. Example output: 
## Article Heading

This paragraph has **bold** and *italic* text.

Another paragraph with a [link](https://www.example.org).
With with_metadata=True, a YAML front-matter block is prepended (same as TXT format). 
from trafilatura import extract, fetch_url

downloaded = fetch_url("https://www.example.org/article")

# markdown output (formatting automatically enabled)
result = extract(downloaded, output_format="markdown")

# markdown with links
result = extract(downloaded, output_format="markdown", include_links=True)

# markdown with metadata header
result = extract(downloaded, output_format="markdown", with_metadata=True)
TXT or CSV output combined with --formatting or --links also triggers Markdown output automatically — no need to switch format manually.

​
JSON

Returns a JSON object. Without metadata, contains only "text" and "comments" fields. With with_metadata=True, all available metadata fields are included. What it includes: Text body, comments, and (optionally) source, source-hostname, title, author, date, excerpt, categories, tags, fingerprint, language, image, pagetype, and more. Example output (without metadata): 
{
  "text": "This is the main article text.",
  "comments": ""
}
Example output (with metadata): 
{
  "title": "Example Article",
  "author": "Jane Doe",
  "url": null,
  "hostname": null,
  "description": "A short description.",
  "sitename": "Example Site",
  "date": "2024-02-01",
  "categories": "",
  "tags": "",
  "fingerprint": "abc123",
  "id": null,
  "license": null,
  "body": null,
  "comments": "",
  "commentsbody": null,
  "raw_text": null,
  "text": "This is the main article text.",
  "language": null,
  "image": null,
  "pagetype": null,
  "filedate": null,
  "source": "https://www.example.org/article",
  "source-hostname": "Example Site",
  "excerpt": "A short description."
}

from trafilatura import extract, fetch_url

downloaded = fetch_url("https://www.example.org/article")

# JSON, text and comments only
result = extract(downloaded, output_format="json")

# JSON with all metadata fields
result = extract(downloaded, output_format="json", with_metadata=True)
The include_formatting parameter has no effect on JSON output. Formatting markup is not included in the "text" field.

​
CSV

Returns a tab-separated values (TSV) string with a fixed column order. All fields are present in every row; missing values are replaced with the string null. Column order: url, id, fingerprint, hostname, title, image, date, text, comments, license, pagetype Example output: 
https://www.example.org/article	null	abc123	example.org	Example Article	null	2024-02-01	This is the main article text.	null	null	article

from trafilatura import extract, fetch_url

downloaded = fetch_url("https://www.example.org/article")

# CSV output
result = extract(downloaded, output_format="csv")

# CSV without comments column
result = extract(downloaded, output_format="csv", include_comments=False)
As with TXT, combining CSV output with --formatting or --links triggers Markdown output rather than plain CSV. To preserve the tabular structure, avoid those flags when using CSV format.

​
HTML

Returns a cleaned, minimal HTML document containing the extracted main content. Metadata fields are emitted as <meta> tags inside a <head> element when with_metadata=True. Available from version 1.11 onwards. What it includes: Main body as semantic HTML, optionally with a <head> block containing metadata <meta> tags. Example output: 
<html>
<body>
<p>This is the main article text.</p>
<p>A second paragraph with more content.</p>
</body>
</html>
With with_metadata=True: 
<html>
<head>
  <meta name="title" content="Example Article"/>
  <meta name="author" content="Jane Doe"/>
  <meta name="date" content="2024-02-01"/>
</head>
<body>
<p>This is the main article text.</p>
</body>
</html>

from trafilatura import extract, fetch_url

downloaded = fetch_url("https://www.example.org/article")

# HTML output
result = extract(downloaded, output_format="html")

# HTML with metadata in <head>
result = extract(downloaded, output_format="html", with_metadata=True)

​
XML

Returns a structured XML document preserving heading hierarchy, paragraph divisions, formatting elements, links, and comments. Metadata is stored as attributes on the root <doc> element. What it includes: <doc> root with metadata attributes, <main> element containing the body, <comments> element, and all preserved structural tags (<p>, <head>, <list>, <quote>, <code>, etc.). Example output: 
<doc sitename="example.org" title="Example Article" author="Jane Doe"
     url="https://www.example.org/article" date="2024-02-01" fingerprint="abc123">
  <main>
    <p>This is the main article text.</p>
    <p>A second paragraph with <hi rend="#b">bold text</hi>.</p>
    <list>
      <item>First item</item>
      <item>Second item</item>
    </list>
  </main>
  <comments/>
</doc>

from trafilatura import extract, fetch_url

downloaded = fetch_url("https://www.example.org/article")
url = "https://www.example.org/article"

# basic XML output
result = extract(downloaded, output_format="xml")

# XML with formatting preserved
result = extract(downloaded, output_format="xml", include_formatting=True)

# XML with links (relative URLs converted to absolute)
result = extract(downloaded, output_format="xml", include_links=True, url=url)

# XML with metadata attributes populated
result = extract(downloaded, output_format="xml", with_metadata=True, url=url)

​
XML-TEI (Text Encoding Initiative)

Returns a fully conformant TEI-XML document. Follows the guidelines of the Text Encoding Initiative and is suitable for archival, scholarly corpora, and digital humanities workflows. The output wraps extracted content in a standard <TEI> document tree with a <teiHeader> and a <text> body. What it includes: Full TEI document with <teiHeader> (metadata), <text><body><div type="entry"> (main content), and <div type="comments"> (comment section). Example output: 
<TEI xmlns="http://www.tei-c.org/ns/1.0">
  <teiHeader>
    <fileDesc>
      <titleStmt>
        <title>Example Article</title>
        <author>Jane Doe</author>
      </titleStmt>
      <publicationStmt>
        <publisher>Example Site</publisher>
        <date>2024-02-01</date>
      </publicationStmt>
      <sourceDesc>
        <bibl>
          <ref target="https://www.example.org/article"/>
        </bibl>
      </sourceDesc>
    </fileDesc>
  </teiHeader>
  <text>
    <body>
      <div type="entry">
        <p>This is the main article text.</p>
      </div>
      <div type="comments"/>
    </body>
  </text>
</TEI>

​
TEI Validation

Validate the output against the TEI standard using --validate-tei on the CLI or tei_validation=True in the extract() call. You can also call trafilatura.xml.validate_tei() directly on a parsed XML tree. 
from trafilatura import extract, fetch_url
from trafilatura.xml import validate_tei
from lxml.etree import fromstring

downloaded = fetch_url("https://www.example.org/article")
url = "https://www.example.org/article"

# TEI output
result = extract(downloaded, output_format="xmltei", url=url)

# TEI output with inline validation (logs result at DEBUG level)
result = extract(downloaded, output_format="xmltei", tei_validation=True, url=url)

# validate a TEI string manually
tree = fromstring(result.encode())
is_valid = validate_tei(tree)
print(is_valid)  # True or False

​
Format Compatibility Notes

TXT/CSV + formatting or links → Markdown: Combining "txt" or "csv" output with include_formatting=True or include_links=True automatically switches the output to Markdown format to accommodate the structural markup.
Metadata in TXT/Markdown: When with_metadata=True is used with "txt" or "markdown" output, a YAML front-matter block is prepended to the document. Fields included are: title, author, url, hostname, description, sitename, date, categories, tags, fingerprint, id, and license.
include_formatting is ignored for JSON: The JSON "text" field always contains plain text. Formatting markup is not included regardless of the include_formatting setting.
Images and links need XML formats: Image and link targets are only fully preserved in XML, XML-TEI, or HTML output. They may not be visible in TXT, Markdown, or CSV output.

Build docs developers (and LLMs) love

Get started for freeTalk to us