Documentation Index
Fetch the complete documentation index at: https://mintlify.com/simonw/LLM/llms.txt
Use this file to discover all available pages before exploring further.
LLM’s plugin system is built on Pluggy, the same framework that powers pytest. Each plugin can implement one or more hook functions decorated with @llm.hookimpl. When LLM starts up, Pluggy discovers all installed plugins and calls each registered hook at the appropriate time. This reference covers every hook LLM exposes, with signatures, parameter descriptions, and working examples.
LLM’s plugin architecture is modelled on Datasette’s plugin system. If you are familiar with writing Datasette plugins, LLM plugins will feel immediately familiar. register_commands(cli)
Adds new subcommands to the llm CLI tool. The cli parameter is the root Click group, so any command you attach becomes available as llm <your-command>.
from llm import hookimpl
import click
@hookimpl
def register_commands(cli):
@cli.command(name="hello-world")
def hello_world():
"Print hello world"
click.echo("Hello world!")
llm --help and can be invoked as:
register_models(register, model_aliases)
Registers one or more language models with LLM. Implement execute() on a subclass of llm.Model and pass an instance to register().
import llm
@llm.hookimpl
def register_models(register):
register(HelloWorld())
class HelloWorld(llm.Model):
model_id = "helloworld"
def execute(self, prompt, stream, response, conversation):
return ["hello world"]
Registering sync and async models together
If your model has both a synchronous and asynchronous implementation, register both in a single call and supply aliases at the same time:
class HelloWorld(llm.Model):
model_id = "helloworld"
def execute(self, prompt, stream, response, conversation):
return ["hello world"]
class AsyncHelloWorld(llm.AsyncModel):
model_id = "helloworld"
async def execute(self, prompt, stream, response, conversation=None):
return ["hello world"]
@llm.hookimpl
def register_models(register):
register(HelloWorld(), AsyncHelloWorld(), aliases=("hw",))
Using model_aliases with trylast
The optional model_aliases parameter is a list of ModelWithAliases objects representing every model registered so far by other plugins. Plugins that use @llm.hookimpl(trylast=True) run after all other plugins and can inspect or modify those registrations:
@llm.hookimpl(trylast=True)
def register_models(register, model_aliases):
# Inspect models registered by other plugins
for mwa in model_aliases:
print(mwa.model.model_id, mwa.aliases)
register(MyModel())
register.
See the Plugin Tutorial for a detailed walkthrough of building a model plugin using this hook. register_embedding_models(register)
Registers embedding models that can be used with llm embed and the embeddings Python API. The model class must extend llm.EmbeddingModel and implement embed_batch().
import llm
@llm.hookimpl
def register_embedding_models(register):
register(HelloWorld())
class HelloWorld(llm.EmbeddingModel):
model_id = "helloworld"
def embed_batch(self, items):
return [[1, 2, 3], [4, 5, 6]]
import llm
def upper(text: str) -> str:
"""Convert text to uppercase."""
return text.upper()
def count_char(text: str, character: str) -> int:
"""Count the number of occurrences of a character in a word."""
return text.count(character)
@llm.hookimpl
def register_tools(register):
register(upper)
# Use name= to override the tool name exposed to the model
register(count_char, name="count_character_in_word")
llm.Toolbox. Pass the class itself (not an instance) to register():
import llm
class Memory(llm.Toolbox):
"""A simple key-value memory store."""
def __init__(self):
self._store = {}
def set(self, key: str, value: str) -> None:
"""Store a value under the given key."""
self._store[key] = value
def get(self, key: str) -> str:
"""Retrieve the value for a key."""
return self._store.get(key, "")
@llm.hookimpl
def register_tools(register):
register(Memory)
If a tool name starts with a capital letter, LLM treats it as a toolbox class rather than a regular tool function.
register_template_loaders(register)
Registers custom template loaders that respond to the llm -t prefix:name syntax. The prefix you register is matched against what the user types before the colon; your loader function receives everything after the colon.
import llm
@llm.hookimpl
def register_template_loaders(register):
register("my-prefix", my_template_loader)
def my_template_loader(template_path: str) -> llm.Template:
"""
Documentation for the template loader goes here. It will be displayed
when users run the 'llm templates loaders' command.
"""
try:
prompt = "This is a sample prompt for {}".format(template_path)
system = "You are an assistant specialized in {}".format(template_path)
return llm.Template(
name=template_path,
prompt=prompt,
system=system,
)
except Exception as e:
raise ValueError(f"Template '{template_path}' could not be loaded: {str(e)}")
llm -t my-prefix:my-template
llm.Template object or raise a ValueError with a descriptive message if the template cannot be found.
Functions defined inside functions: blocks in templates loaded via this hook will not be executed, to prevent arbitrary code execution from plugins that load templates from remote sources.
register_fragment_loaders(register)
Registers fragment loaders that respond to the llm -f prefix:argument syntax. Unlike template loaders, multiple fragment loaders can be stacked in a single prompt invocation.
A fragment loader returns one or more llm.Fragment or llm.Attachment objects (or a mixed list of both):
import llm
@llm.hookimpl
def register_fragment_loaders(register):
register("my-fragments", my_fragment_loader)
def my_fragment_loader(argument: str) -> llm.Fragment:
"""
Documentation for the fragment loader goes here. It will be displayed
when users run the 'llm fragments loaders' command.
"""
try:
fragment = "Fragment content for {}".format(argument)
source = "my-fragments:{}".format(argument)
return llm.Fragment(fragment, source)
except Exception as ex:
raise ValueError(
f"Fragment 'my-fragments:{argument}' could not be loaded: {str(ex)}"
)
Returning multiple fragments and attachments
Loaders can return a list mixing llm.Fragment and llm.Attachment objects. LLM treats this exactly as if the user had passed multiple -f arguments:
def my_fragment_loader(argument: str) -> list:
"""Return multiple fragments and an image attachment."""
return [
llm.Fragment("Fragment 1 content", f"my-fragments:{argument}"),
llm.Fragment("Fragment 2 content", f"my-fragments:{argument}"),
llm.Attachment(path="/path/to/image.png"),
]
llm.Fragment takes a required string (the content) and an optional source string used as debug information — typically a file path or URL.
Invoke a fragment loader:
llm -f my-fragments:argument
Returning individual fragments per file (rather than concatenating everything) means LLM’s deduplication cache can avoid storing multiple copies of a directory when only a single file has changed.