Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/simonw/LLM/llms.txt

Use this file to discover all available pages before exploring further.

The plugins below extend LLM with additional models, tools, embedding backends, CLI commands, and content loaders. Install any of them with llm install <package-name> — see Installing Plugins for details. Plugins are grouped by the kind of capability they add.
These plugins let you run LLMs directly on your own machine without sending data to an external API.
PluginDescription
llm-ollamaAdds support for local models managed by Ollama.
llm-mlx(Mac only) Uses Apple’s MLX framework for extremely fast access to a wide range of local models.
llm-ggufUses llama.cpp to run any model published in the GGUF format.
llm-llamafileConnects to local models running via llamafile.
llm-gpt4allRuns GPT4All optimised models locally, including Vicuna, Orca, Falcon, and MPT variants.
llm-mlcRuns models released by the MLC project, with GPU acceleration on Apple Silicon.
llm-mpt30bAdds support for the MPT-30B local model.
Example — install and use Ollama models:
llm install llm-ollama
llm models
llm -m ollama/llama3 'Explain how neural networks work'
Example — install and use a GGUF model:
llm install llm-gguf
llm gguf download-model https://example.com/model.gguf
llm -m orca-mini-3b-gguf2-q4_0 'What is the capital of France?'
These plugins connect LLM to externally hosted model APIs. You’ll need an API key for each service.
PluginProvider
llm-anthropicAnthropic — Claude 3 family, Claude 3.5 Sonnet, and beyond
llm-geminiGoogle — Gemini language and embedding models
llm-mistralMistral AI — language and embedding models
llm-groqGroq — ultra-fast inference
llm-grokxAI — Grok models via the xAI API
llm-command-rCohere — Command R and Command R Plus
llm-cohereCohere — cohere-generate and cohere-summarize models
llm-deepseekDeepSeek — DeepSeek-Chat and DeepSeek-Coder
llm-perplexityPerplexity Labs — online search-augmented models
llm-rekaReka AI — the full Reka model family
llm-replicateReplicate — Llama 2 and other hosted models
llm-openrouterOpenRouter — unified access to many providers
llm-fireworksFireworks AI — fast open-model inference
llm-togetherTogether AI — large catalogue of open licensed models
llm-bedrockAmazon Bedrock — Nova models by Amazon
llm-bedrock-anthropicAmazon Bedrock — Claude and Claude Instant via Bedrock
llm-bedrock-metaAmazon Bedrock — Llama 2 and Llama 3 via Bedrock
llm-anyscale-endpointsAnyscale Endpoints — Llama 2 70B and more
llm-lambda-labsLambda Labs — Nous Hermes 3 series
llm-veniceVenice AI — privacy-focused, uncensored models including Llama 3.1 405B
Example — install and use Anthropic’s Claude:
llm install llm-anthropic
llm keys set anthropic
llm -m claude-3-5-sonnet-latest 'Summarise the history of the internet'
If a provider exposes an OpenAI-compatible API, you can often configure LLM to use it directly without installing a dedicated plugin.
Tool plugins expose callable functions that a model can invoke during a conversation — for example, running code, querying a database, or searching the web.
PluginWhat it provides
llm-tools-simpleevalSafe expression evaluation for maths and simple logic
llm-tools-quickjsSandboxed QuickJS JavaScript interpreter with persistent state between calls
llm-tools-sqliteRead-only SQL queries against local SQLite databases
llm-tools-datasetteSQL queries against a remote Datasette instance
llm-tools-exaWeb search and question-answering via exa.ai
llm-tools-ragSemantic search over your LLM embedding collections for simple RAG
Example — let a model run JavaScript:
llm install llm-tools-quickjs
llm -T llm_tools_quickjs 'Calculate the first 10 Fibonacci numbers'
Introduced in LLM 0.24, these plugins add custom -f prefix:value fragment loaders and -t prefix:value template loaders, making it easy to pull external content directly into a prompt.Fragment loaders (-f prefix:value)
PluginWhat it loads
llm-fragments-githubEntire GitHub repositories or individual issue threads
llm-fragments-pdfPDF files converted to Markdown via PyMuPDF4LLM
llm-fragments-pypiPyPI package descriptions and metadata
llm-fragments-site-textWebsites converted to Markdown via Trafilatura
llm-fragments-readerAny URL processed through the Jina Reader API
llm-hacker-newsHacker News threads imported as conversation fragments
llm-video-framesVideo files converted to JPEG frames via ffmpeg for vision models
Template loaders (-t prefix:value)
PluginWhat it loads
llm-templates-githubPrompt templates shared on GitHub
llm-templates-fabricPrompts from the Fabric collection
Examples:
# Summarise an entire GitHub repo
llm install llm-fragments-github
llm -f github:simonw/files-to-prompt 'explain this codebase'

# Load a GitHub issue thread
llm -f issue:https://github.com/simonw/llm/issues/1 'summarise this discussion'

# Ask a question about a PDF
llm install llm-fragments-pdf
llm -f pdf:report.pdf "what are the key findings?"

# Use a Fabric prompt template
llm install llm-templates-fabric
cat setup.py | llm -t fabric:explain_code

# Extract frames from a video and describe it
llm install llm-video-frames
llm -f video-frames:demo.mp4 'describe the key scenes in this video'
Embedding model plugins add backends for generating vector embeddings — used with llm embed and the llm embed-multi / llm similar commands.
PluginModels provided
llm-sentence-transformersAny model from the sentence-transformers library
llm-clipOpenAI’s CLIP model — embeds images and text in the same vector space
llm-embed-jinaJina AI’s 8K text embedding models
llm-embed-onnxSeven embedding models running via the ONNX runtime
Example — embed and search text with sentence-transformers:
llm install llm-sentence-transformers
llm embed-models          # list available embedding models
llm embed -m sentence-transformers/all-MiniLM-L6-v2 -c "Hello world"
Example — image search with CLIP:
llm install llm-clip
llm embed -m clip -c "a photo of a cat"
These plugins bolt new subcommands onto the llm CLI.
PluginCommand addedWhat it does
llm-cmdllm cmdGenerates a shell command from a prompt, pre-fills your shell input, and lets you review before running
llm-cmd-compShell keybindingLaunches an interactive chat to build a command; inserts the result into your shell command line
llm-pythonllm pythonOpens a Python REPL inside LLM’s virtual environment — handy for scripting and debugging
llm-clusterllm clusterClusters a collection of embeddings and generates summary labels using an LLM
llm-jqllm jqPipes JSON and a natural-language description into jq and executes the generated program
Example — generate and run a shell command:
llm install llm-cmd
llm cmd 'find all PDF files modified in the last 7 days'
# Review the generated command, then press Enter to run or Ctrl+C to cancel
Example — cluster embeddings:
llm install llm-cluster
llm cluster my-collection 10
Experimental or novelty plugins — also useful as learning resources for plugin authors.
PluginDescription
llm-markovA simple model that generates text using a Markov chain. Used as the example in the Writing a plugin to support a new model tutorial.
llm install llm-markov
llm -m markov 'hello world'

Build docs developers (and LLMs) love

Get started for freeTalk to us