Skip to main content

​
Overview

PromptExecutor is responsible for executing prompts against language models, handling streaming responses, and managing LLM interactions. It serves as the bridge between the agent framework and underlying LLM providers.
In agent contexts, the executor is wrapped by ContextualPromptExecutor which adds feature pipeline integration and event interception.

​
Class Definition

public abstract class PromptExecutor : Closeable

​
Core Methods

​
execute

Executes a prompt and returns the response messages. 
public abstract suspend fun execute(
    prompt: Prompt,
    model: LLModel,
    tools: List<ToolDescriptor>
): List<Message.Response>
​
prompt
Prompt
required
The prompt to send to the language model, containing messages and configuration
​
model
LLModel
required
The language model to use for execution
​
tools
List<ToolDescriptor>
required
List of tool descriptors available for the LLM to use
​
return
List<Message.Response>
List of response messages from the language model. Typically contains one response, but may contain multiple for choice-based models.

​
executeStreaming

Executes a streaming call to the language model. 
public abstract fun executeStreaming(
    prompt: Prompt,
    model: LLModel,
    tools: List<ToolDescriptor>
): Flow<StreamFrame>
​
prompt
Prompt
required
The prompt to send to the language model
​
model
LLModel
required
The language model to use for streaming
​
tools
List<ToolDescriptor>
required
List of available tool descriptors
​
return
Flow<StreamFrame>
A Flow of StreamFrame objects representing the streaming response. Each frame contains partial response data.

​
executeMultipleChoices

Executes a prompt with multiple choice generation. 
public abstract suspend fun executeMultipleChoices(
    prompt: Prompt,
    model: LLModel,
    tools: List<ToolDescriptor>
): List<LLMChoice>
​
prompt
Prompt
required
The prompt to send to the language model
​
model
LLModel
required
The language model to use
​
tools
List<ToolDescriptor>
required
List of available tool descriptors
​
return
List<LLMChoice>
List of choice objects, each containing a sequence of response messages

​
moderate

Executes content moderation on a prompt. 
public abstract suspend fun moderate(
    prompt: Prompt,
    model: LLModel
): ModerationResult
​
prompt
Prompt
required
The prompt to moderate
​
model
LLModel
required
The language model to use for moderation
​
return
ModerationResult
The moderation result indicating whether content violates policies

​
models

Retrieves available language models. 
public abstract suspend fun models(): List<LLModel>
​
return
List<LLModel>
List of available language models from the provider

​
Schema Generation

​
getStandardJsonSchemaGenerator

Gets a JSON schema generator for structured outputs. 
public abstract fun getStandardJsonSchemaGenerator(
    model: LLModel
): StandardJsonSchemaGenerator
​
model
LLModel
required
The language model to generate schemas for
​
return
StandardJsonSchemaGenerator
Schema generator for creating JSON schemas from Kotlin types

​
getBasicJsonSchemaGenerator

Gets a basic JSON schema generator. 
public abstract fun getBasicJsonSchemaGenerator(
    model: LLModel
): BasicJsonSchemaGenerator
​
model
LLModel
required
The language model to generate schemas for
​
return
BasicJsonSchemaGenerator
Basic schema generator for simpler JSON schema needs

​
ContextualPromptExecutor

Wraps PromptExecutor with feature pipeline integration: 
public class ContextualPromptExecutor(
    private val executor: PromptExecutor,
    private val context: AIAgentContext
) : PromptExecutor()
The contextual wrapper:
  • Triggers pipeline events before and after LLM calls
  • Allows features to intercept and modify prompts
  • Provides logging and debugging information
  • Integrates with agent execution lifecycle

​
Usage Examples

​
Basic Execution

val executor = PromptExecutor.create(
    apiKey = "your-api-key",
    provider = LLMProvider.OPENAI
)

val prompt = Prompt {
    user("What is the capital of France?")
}

val model = LLModel.GPT4

val responses = executor.execute(
    prompt = prompt,
    model = model,
    tools = emptyList()
)

println(responses.first().content)

executor.close()

​
Streaming Response

val executor = PromptExecutor.create(
    apiKey = apiKey,
    provider = LLMProvider.ANTHROPIC
)

val prompt = Prompt {
    user("Tell me a story")
}

executor.executeStreaming(
    prompt = prompt,
    model = LLModel.CLAUDE_3_5_SONNET,
    tools = emptyList()
).collect { frame ->
    when (frame) {
        is StreamFrame.Content -> print(frame.text)
        is StreamFrame.Done -> println("\nDone!")
        else -> { /* handle other frames */ }
    }
}

executor.close()

​
With Tools

val tools = listOf(
    SearchTool().descriptor,
    CalculatorTool().descriptor
)

val prompt = Prompt {
    system("You are a helpful assistant with access to tools")
    user("What is the weather in Paris?")
}

val responses = executor.execute(
    prompt = prompt,
    model = model,
    tools = tools
)

// Handle potential tool calls
responses.forEach { response ->
    when (response) {
        is Message.Assistant.ToolCall -> {
            println("LLM wants to call: ${response.toolCalls}")
        }
        is Message.Assistant.Text -> {
            println("Response: ${response.content}")
        }
    }
}

​
Multiple Choices

val prompt = Prompt {
    user("Generate creative names for a coffee shop")
}

val choices = executor.executeMultipleChoices(
    prompt = prompt,
    model = model.copy(numberOfChoices = 3),
    tools = emptyList()
)

choices.forEachIndexed { index, choice ->
    println("Choice ${index + 1}: ${choice.first().content}")
}

​
Content Moderation

val prompt = Prompt {
    user(userInput)
}

val moderationResult = executor.moderate(
    prompt = prompt,
    model = model
)

if (moderationResult.flagged) {
    println("Content flagged: ${moderationResult.categories}")
} else {
    // Safe to proceed
    val response = executor.execute(prompt, model, tools)
}

​
In Agent Context

When used within an agent, the executor is automatically wrapped: 
class MyStrategy : AIAgentGraphStrategy<String, String> {
    override val name = "my-strategy"
    
    override suspend fun execute(
        context: AIAgentGraphContext,
        input: String
    ): String? {
        // The context.llm uses ContextualPromptExecutor internally
        // This triggers pipeline events and feature interception
        val response = context.llm.prompt {
            user(input)
        }.execute()
        
        return response.content
    }
}

​
Provider-Specific Executors

Create executors for specific providers: 
// OpenAI
val openAIExecutor = OpenAIPromptExecutor(
    apiKey = openAIKey,
    organization = "org-id"
)

// Anthropic
val anthropicExecutor = AnthropicPromptExecutor(
    apiKey = anthropicKey
)

// Google (Gemini)
val geminiExecutor = GeminiPromptExecutor(
    apiKey = geminiKey
)

// OpenRouter
val openRouterExecutor = OpenRouterPromptExecutor(
    apiKey = openRouterKey
)

// Ollama (local)
val ollamaExecutor = OllamaPromptExecutor(
    baseUrl = "http://localhost:11434"
)

​
Feature Pipeline Integration

The ContextualPromptExecutor integrates with the agent pipeline: 
// Features can intercept LLM calls
pipeline.interceptLLMCallStarting(feature) { eventContext ->
    // Modify the prompt before execution
    eventContext.context.llm.prompt = Prompt {
        system("Enhanced system prompt")
        messages(eventContext.prompt.messages)
    }
}

pipeline.interceptLLMCallCompleted(feature) { eventContext ->
    // Process responses after execution
    eventContext.responses.forEach { response ->
        println("LLM said: ${response.content}")
    }
}

​
Best Practices

Resource Management
  • Always close the executor when done using executor.close()
  • Use use { } blocks for automatic cleanup
  • Executors maintain HTTP clients and connections
  • Don’t create multiple executors unnecessarily
Error Handling
  • LLM calls can fail due to network issues, rate limits, or API errors
  • Implement retry logic for transient failures
  • Handle timeout exceptions appropriately
  • Validate API keys and credentials before use

​
Testing

Mock executors for testing: 
val mockExecutor = object : PromptExecutor() {
    override suspend fun execute(
        prompt: Prompt,
        model: LLModel,
        tools: List<ToolDescriptor>
    ): List<Message.Response> {
        return listOf(
            Message.Assistant.Text("Mocked response")
        )
    }
    
    // Implement other abstract methods...
}
Or use the testing utilities: 
val mockExecutor = getMockExecutor(toolRegistry, eventHandler) {
    mockLLMAnswer("Hello!") onRequestContains "Hello"
    mockLLMAnswer("Default response").asDefaultResponse
}

​
Source Reference

Defined in: agents-core/src/commonMain/kotlin/ai/koog/agents/core/feature/ContextualPromptExecutor.kt

Build docs developers (and LLMs) love

Get started for freeTalk to us