Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/simonw/LLM/llms.txt

Use this file to discover all available pages before exploring further.

LLM ships a set of utility functions designed specifically for plugin authors. These cover the most common needs: looking up API keys stored by the user, finding the right place to persist plugin data, raising user-friendly errors, and generating fake response objects for tests. This page documents each utility with its signature, parameters, and a practical example.

​
llm.get_key()

Retrieves an API key or other secret from LLM’s key registry. Returns the key as a string, or None if it cannot be resolved. 
import llm

github_key = llm.get_key(alias="github")

​
Falling back to an environment variable

Pass env= to check an environment variable if the key isn’t in the registry: 
github_key = llm.get_key(alias="github", env="GITHUB_TOKEN")

​
Accepting user-supplied input

When your plugin lets users type a key directly (rather than pre-configuring it), pass the raw input as input=. If the input matches a stored alias it is looked up; otherwise it is used verbatim: 
github_key = llm.get_key(input=input_from_user, alias="github", env="GITHUB_TOKEN")
The resolution order is:
  1. input value, if it matches a key alias in keys.json
  2. input value verbatim, if it doesn’t match an alias
  3. Stored key for alias, if configured
  4. Environment variable named by env
  5. None
An older positional-argument calling convention is still supported for backwards compatibility, but the keyword argument form shown here is strongly preferred.

​
llm.user_dir()

Returns the path to LLM’s configuration directory as a pathlib.Path object, creating the directory if it doesn’t already exist. On macOS this is ~/Library/Application Support/io.datasette.llm. The exact location varies by operating system. Plugins should store their own data in a subdirectory of this directory: 
import llm

user_dir = llm.user_dir()
plugin_dir = user_dir / "my-plugin"
plugin_dir.mkdir(exist_ok=True)
data_path = plugin_dir / "plugin-data.db"
Always create your subdirectory with exist_ok=True. The user directory itself is guaranteed to exist when user_dir() returns, but subdirectories are not.

​
llm.ModelError

Raise llm.ModelError from inside execute() to surface a clean error message to the user without a Python traceback. LLM’s CLI layer catches this exception and displays the message directly: 
import llm

def execute(self, prompt, stream, response, conversation):
    if not self._model_downloaded():
        raise llm.ModelError(
            "MPT model not installed - try running 'llm mpt30b download'"
        )
    ...
Use ModelError for expected failure conditions — missing dependencies, invalid configuration, quota exceeded — where a stack trace would be confusing rather than helpful.

​
AsyncResponse.fake()

Creates a pre-built AsyncResponse object for use in tests. This is useful when testing code that consumes async responses, such as conversation history builders. The signature is: 
@classmethod
def fake(
    cls,
    model: llm.AsyncModel,
    prompt: str,
    *attachments,
    system: str,
    response: str,
) -> AsyncResponse:
    ...
Example usage: 
import llm

async def test_async_conversation():
    model = llm.get_async_model("my-async-model")
    fake_response = llm.AsyncResponse.fake(
        model,
        "What is 2+2?",
        system="You are a maths tutor.",
        response="The answer is 4.",
    )
    assert await fake_response.text() == "The answer is 4."
The returned object has _done set to True and its _chunks pre-populated with the provided response string, so it behaves like a completed async response without any real model invocation.

Build docs developers (and LLMs) love

Get started for freeTalk to us