Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/simonw/LLM/llms.txt

Use this file to discover all available pages before exploring further.

LLM is an open-source project and welcomes contributions of all kinds — bug fixes, new features, documentation improvements, and plugin development. This guide walks you through setting up a local development environment, running the test suite, updating recorded API interactions, and shipping a new release.

​
Development Environment

1

Clone the repository

git clone https://github.com/simonw/llm
cd llm
2

Run tests (uv installs dependencies automatically)

LLM uses uv for dependency management. On the first run, uv automatically creates a virtual environment and installs all required packages, including development dependencies. Just run:
uv run pytest
You can also invoke your development copy of llm directly via uv run:
uv run llm --help

​
Running Tests

Run the full test suite with: 
uv run pytest
You can pass any standard pytest arguments — for example uv run pytest tests/test_cli.py -k test_prompt -v to run a specific test with verbose output.

​
Updating Recorded HTTP API Interactions

LLM uses pytest-recording (built on VCR.py) to record real OpenAI API responses and replay them in tests, and syrupy to capture snapshots of the results. When you add a new test that calls the OpenAI API, capture the API response and its snapshot in one step: 
PYTEST_OPENAI_API_KEY="$(llm keys get openai)" uv run pytest \
  --record-mode once \
  --snapshot-update
After running this command:
  1. New VCR cassette files will be written under tests/cassettes/.
  2. New snapshot files will be written under tests/__snapshots__/.
Review the generated snapshots in tests/__snapshots__/ carefully before committing them. Make sure the captured output looks correct and does not contain any sensitive data.

​
Debugging Tricks

The default OpenAI plugin can log the exact HTTP requests and responses it sends and receives. Enable it with:
LLM_OPENAI_SHOW_RESPONSES=1 uv run llm -m chatgpt \
  'three word slogan for an otter-run bakery'
To see a cleaner, non-streamed version of the response body:
LLM_OPENAI_SHOW_RESPONSES=1 uv run llm -m chatgpt --no-stream \
  'three word slogan for an otter-run bakery'
By default LLM suppresses errors during prompt execution. Set LLM_RAISE_ERRORS=1 to have them propagate as Python exceptions instead:
LLM_RAISE_ERRORS=1 python -i -m llm 'my prompt'
If an error occurs, Python drops you into an interactive session. From there you can post-mortem debug:
import pdb; pdb.pm()
When running tests, add --pdb to drop into the debugger at the point of any failure:
uv run pytest --pdb

​
Documentation

LLM’s documentation uses MyST — written in Markdown and rendered with Sphinx.

​
Building docs locally

just docs
This starts a live-preview server using sphinx-autobuild. Changes to .md files are reflected in the browser automatically.
You need Just installed to run just commands. Install it with brew install just on macOS or see the Just installation guide for other platforms.

​
Updating CLI help output in the docs

The --help examples in the documentation are auto-generated using Cog. After changing any CLI flags or help text, regenerate the docs: 
just cog

​
Release Process

1

Update the changelog

Add an entry for the new version to docs/changelog.md describing all changes.
2

Bump the version number

Update the version in pyproject.toml:
[project]
version = "0.XX.0"
3

Regenerate cog output

Run just cog to update docs/fragments.md and any other cog-managed files with the new version number:
just cog
4

Create a GitHub Release

Create a new GitHub release tagged with the new version number. The release notes should mirror the changelog entry.
5

Wait for PyPI

GitHub Actions will automatically publish the package to PyPI when the release is created. Wait for that workflow to complete.
6

Update the Homebrew tap

Run the regenerate.yaml workflow in the simonw/homebrew-llm repository to update the Homebrew tap formula to the new version.

Build docs developers (and LLMs) love

Get started for freeTalk to us