LlamaIndex Adapter
LlamaIndex is a framework for building LLM-powered applications. LlamaIndex helps you ingest, structure, and access private or domain-specific data. LlamaIndex.TS offers the core features of LlamaIndex for Python for popular runtimes like Node.js (official support), Vercel Edge Functions (experimental), and Deno (experimental).
Installation
Tab Title
Tab Title
Tab Title
llamaindex is a required peer dependency.
Features
- Transform LlamaIndex ChatEngine and QueryEngine streams to AI SDK
UIMessageStream
- Seamless integration with AI SDK UI components like
useCompletion
- Support for RAG (Retrieval Augmented Generation) workflows
- Compatible with LlamaIndex’s document processing and indexing capabilities
Example: Completion
Here is a basic example that uses both AI SDK and LlamaIndex together with the Next.js App Router.
The AI SDK @ai-sdk/llamaindex package uses the stream result from calling the chat method on a LlamaIndex ChatEngine or the query method on a LlamaIndex QueryEngine to pipe text to the client.
import { OpenAI, SimpleChatEngine } from 'llamaindex';
import { toUIMessageStream } from '@ai-sdk/llamaindex';
import { createUIMessageStreamResponse } from 'ai';
export const maxDuration = 60;
export async function POST(req: Request) {
const { prompt } = await req.json();
const llm = new OpenAI({ model: 'gpt-4o' });
const chatEngine = new SimpleChatEngine({ llm });
const stream = await chatEngine.chat({
message: prompt,
stream: true,
});
return createUIMessageStreamResponse({
stream: toUIMessageStream(stream),
});
}
useCompletion method in the page component to handle the completion:
'use client';
import { useCompletion } from '@ai-sdk/react';
export default function Chat() {
const { completion, input, handleInputChange, handleSubmit } =
useCompletion();
return (
<div>
{completion}
<form onSubmit={handleSubmit}>
<input value={input} onChange={handleInputChange} />
</form>
</div>
);
}
Example: RAG with QueryEngine
LlamaIndex excels at building RAG applications. Here’s an example using a QueryEngine with document indexing:
import {
OpenAI,
VectorStoreIndex,
SimpleDirectoryReader,
} from 'llamaindex';
import { toUIMessageStream } from '@ai-sdk/llamaindex';
import { createUIMessageStreamResponse } from 'ai';
export const maxDuration = 60;
// Initialize once (consider caching in production)
let queryEngine: any = null;
async function getQueryEngine() {
if (!queryEngine) {
// Load documents from a directory
const reader = new SimpleDirectoryReader();
const documents = await reader.loadData('./data');
// Create index from documents
const index = await VectorStoreIndex.fromDocuments(documents);
// Create query engine
queryEngine = index.asQueryEngine({
llm: new OpenAI({ model: 'gpt-4o' }),
});
}
return queryEngine;
}
export async function POST(req: Request) {
const { prompt } = await req.json();
const engine = await getQueryEngine();
const stream = await engine.query({
query: prompt,
stream: true,
});
return createUIMessageStreamResponse({
stream: toUIMessageStream(stream),
});
}
Example: Chat with Context
Build a conversational interface with document context:
import {
OpenAI,
VectorStoreIndex,
ContextChatEngine,
SimpleDirectoryReader,
} from 'llamaindex';
import { toUIMessageStream } from '@ai-sdk/llamaindex';
import { createUIMessageStreamResponse } from 'ai';
export const maxDuration = 60;
let chatEngine: any = null;
async function getChatEngine() {
if (!chatEngine) {
const reader = new SimpleDirectoryReader();
const documents = await reader.loadData('./data');
const index = await VectorStoreIndex.fromDocuments(documents);
// Create a chat engine with context
chatEngine = new ContextChatEngine({
retriever: index.asRetriever(),
llm: new OpenAI({ model: 'gpt-4o' }),
});
}
return chatEngine;
}
export async function POST(req: Request) {
const { prompt } = await req.json();
const engine = await getChatEngine();
const stream = await engine.chat({
message: prompt,
stream: true,
});
return createUIMessageStreamResponse({
stream: toUIMessageStream(stream),
});
}
useCompletion hook on the client:
'use client';
import { useCompletion } from '@ai-sdk/react';
export default function ChatWithContext() {
const { completion, input, handleInputChange, handleSubmit, isLoading } =
useCompletion({
api: '/api/chat',
});
return (
<div>
<div className="response">
{completion || 'Ask a question about your documents...'}
</div>
<form onSubmit={handleSubmit}>
<input
value={input}
onChange={handleInputChange}
disabled={isLoading}
placeholder="What would you like to know?"
/>
<button type="submit" disabled={isLoading}>
{isLoading ? 'Thinking...' : 'Ask'}
</button>
</form>
</div>
);
}
API Reference
toUIMessageStream(stream)
Converts a LlamaIndex ChatEngine or QueryEngine stream to an AI SDK UIMessageStream.
import { toUIMessageStream } from '@ai-sdk/llamaindex';
import { createUIMessageStreamResponse } from 'ai';
const stream = await chatEngine.chat({
message: prompt,
stream: true,
});
return createUIMessageStreamResponse({
stream: toUIMessageStream(stream),
});
stream: AsyncIterable - Stream from LlamaIndex ChatEngine or QueryEngine
Returns: ReadableStream<UIMessageChunk>
Integration with LlamaIndex Features
The adapter works seamlessly with LlamaIndex’s powerful features:
Document Loaders
- Load documents from various sources (files, URLs, databases)
- Support for multiple file formats (PDF, Markdown, JSON, etc.)
- Custom document readers
Vector Stores
- In-memory vector storage
- Integration with external vector databases
- Efficient similarity search
Retrievers
- Vector similarity retrieval
- Keyword-based retrieval
- Hybrid retrieval strategies
Query Engines
- Simple query engine for basic RAG
- Sub-question query engine for complex queries
- Custom query engines
Chat Engines
- Simple chat engine
- Context chat engine with retrieval
- Condense question chat engine
More Examples
create-llama is the easiest way to get started with LlamaIndex. It uses the AI SDK to connect to LlamaIndex in all its generated code.
Learn More