The LLMObs SDK is available as tracer.llmobs. It provides methods for tracing LLM operations, annotating spans, and submitting evaluation metrics.
const tracer = require ( 'dd-trace' ). init ({ llmobs: { mlApp: 'my-llm-app' , }, }) const llmobs = tracer . llmobs
Properties enabledWhether LLM Observability tracing is currently enabled.
Methods enable(options)Enables LLM Observability tracing programmatically.
enable ( options : LLMObsEnableOptions ): void
options
LLMObsEnableOptions
required
Enable options. Show LLMObsEnableOptions properties
The name of your ML application. Also configurable via DD_LLMOBS_ML_APP.
Set to true to disable sending data that requires a Datadog Agent. Also configurable via DD_LLMOBS_AGENTLESS_ENABLED.
tracer . llmobs . enable ({ mlApp: 'my-chat-app' })
disable()Disables LLM Observability tracing.
trace(options, fn)Instruments a function by automatically creating an LLMObs span that is activated on its scope.
trace < T >( options : LLMObsNamedSpanOptions , fn : ( span : Span , done : ( error ?: Error ) => void ) => T ): T
options
LLMObsNamedSpanOptions
required
LLMObs span options. Show LLMObsNamedSpanOptions properties
The name of the traced operation.
LLM Observability span kind. One of: agent, workflow, task, tool, retrieval, embedding, llm.
The invoked model name. Only used on llm and embedding spans.
The model provider (e.g., openai). Only used on llm and embedding spans. Defaults to custom.
The user session ID. Required for session tracking.
ML application name. Overrides the default set at initialization.
The function to instrument. Receives the active span and an optional done callback.
const response = await tracer . llmobs . trace ( { kind: 'llm' , name: 'openai.chat' , modelName: 'gpt-4o' , modelProvider: 'openai' , sessionId: 'user-session-123' , }, async ( span ) => { const result = await openai . chat . completions . create ({ model: 'gpt-4o' , messages: [{ role: 'user' , content: 'Hello!' }], }) tracer . llmobs . annotate ( span , { inputData: [{ role: 'user' , content: 'Hello!' }], outputData: { role: 'assistant' , content: result . choices [ 0 ]. message . content }, metrics: { inputTokens: result . usage . prompt_tokens , outputTokens: result . usage . completion_tokens , totalTokens: result . usage . total_tokens , }, }) return result } )
wrap(options, fn)Wraps a function so that an LLMObs span is created automatically each time the function is called.
wrap < T = ( ... args : any []) => any >( options : LLMObsNamelessSpanOptions , fn : T ) : T
options
LLMObsNamelessSpanOptions
required
LLMObs span options. The name field is optional — if omitted, the function name is used.
A wrapped function with the same signature.
async function callLLM ( prompt ) { const result = await openai . chat . completions . create ({ model: 'gpt-4o' , messages: [{ role: 'user' , content: prompt }], }) return result . choices [ 0 ]. message . content } const tracedCallLLM = tracer . llmobs . wrap ( { kind: 'llm' , modelName: 'gpt-4o' , modelProvider: 'openai' }, callLLM ) const response = await tracedCallLLM ( 'What is 2 + 2?' )
annotate(options) / annotate(span, options)Sets inputs, outputs, tags, metadata, and metrics on a given LLMObs span. With the exception of tags, this method overwrites any existing values for the provided fields.
annotate ( options : AnnotationOptions ): void annotate ( span : Span | undefined , options : AnnotationOptions ): void
The span to annotate. Defaults to the current active LLMObs span if not provided.
options
AnnotationOptions
required
Annotation data. Show AnnotationOptions properties
inputData
string | Message | Message[] | Document | Document[] | object
Input data. For llm spans: a string or {content, role} message(s). For embedding spans: a string or {text, ...} object(s). For other spans: any JSON-serializable value.
outputData
string | Message | Message[] | Document | Document[] | object
Output data. For llm spans: a string or {content, role} message. For retrieval spans: {name, id, text, score} object(s). For other spans: any JSON-serializable value.
JSON-serializable key-value metadata relevant to the operation.
Numeric key-value metrics, such as { inputTokens, outputTokens, totalTokens }.
Key-value tag pairs. Tags are merged (not replaced) with existing tags.
Prompt template object. Only used on llm spans.
tracer . llmobs . trace ({ kind: 'llm' , name: 'my-llm' , modelName: 'gpt-4o' , modelProvider: 'openai' }, ( span ) => { tracer . llmobs . annotate ( span , { inputData: [ { content: 'You are a helpful assistant.' , role: 'system' }, { content: 'What is the capital of France?' , role: 'user' }, ], outputData: { content: 'Paris.' , role: 'assistant' }, metadata: { temperature: 0.7 , max_tokens: 100 }, tags: { environment: 'production' }, metrics: { inputTokens: 20 , outputTokens: 5 , totalTokens: 25 }, }) })
flush()Flushes any remaining spans and evaluation metrics to LLM Observability.
// At process shutdown process . on ( 'beforeExit' , () => { tracer . llmobs . flush () })
exportSpan(span?)Returns an object containing the span ID and trace ID for the given span. Used with submitEvaluation().
exportSpan ( span ?: Span ): ExportedLLMObsSpan
The span to export. Defaults to the current active LLMObs span.
An object with spanId and traceId strings.
const spanContext = tracer . llmobs . exportSpan ( span ) console . log ( spanContext . spanId ) console . log ( spanContext . traceId )
submitEvaluation(spanContext, options)Submits a custom evaluation metric for a span.
submitEvaluation ( spanContext : ExportedLLMObsSpan , options : EvaluationOptions ): void
spanContext
ExportedLLMObsSpan
required
The exported span context from exportSpan().
options
EvaluationOptions
required
Evaluation metric data. Show EvaluationOptions properties
The name of the evaluation metric.
One of: categorical, score, boolean, json.
value
string | number | boolean | object
required
The metric value. Must match the metricType: string for categorical, number for score, boolean for boolean, object for json.
String key-value tags to attach to the metric.
ML application name override.
Timestamp in milliseconds when the evaluation was generated.
Reasoning for the evaluation result.
Pass/fail assessment. One of: pass, fail.
const spanCtx = tracer . llmobs . exportSpan ( span ) tracer . llmobs . submitEvaluation ( spanCtx , { label: 'faithfulness' , metricType: 'score' , value: 0.95 , tags: { evaluator: 'gpt-4' }, reasoning: 'Response accurately reflects the source material' , assessment: 'pass' , })
annotationContext(options, fn)Annotates all spans (including auto-instrumented spans) created within the callback with the provided context options.
annotationContext < T >( options : AnnotationContextOptions , fn : () => T ): T
options
AnnotationContextOptions
required
Context annotation options. Show AnnotationContextOptions properties
Key-value tags to apply to all spans within the context.
Override the span name for any spans annotated within the context.
Prompt template to apply to llm spans within the context.
tracer . llmobs . annotationContext ( { tags: { session: 'abc123' , user_tier: 'premium' } }, () => { // All LLMObs spans created in here get the tags above processRequest () } )
routingContext(options, fn)Executes a function within a routing context that directs all LLMObs spans to a specific Datadog organization.
routingContext < T >( options : RoutingContextOptions , fn : () => T ): T
options
RoutingContextOptions
required
Routing context options. Show RoutingContextOptions properties
The Datadog API key for the target organization.
The Datadog site for the target organization (e.g., datadoghq.eu).
tracer . llmobs . routingContext ( { ddApiKey: 'customer-api-key' , ddSite: 'datadoghq.eu' }, () => { // Spans sent to customer's Datadog org processCustomerRequest () } )
registerProcessor(processor)Registers a processor function that is called on each LLMObs span before it is sent. The processor can modify the span or return null to drop it.
registerProcessor ( processor : ( span : LLMObservabilitySpan ) => LLMObservabilitySpan | null ): void
A function receiving an LLMObservabilitySpan. Return the (possibly modified) span to include it, or null to drop it.
tracer . llmobs . registerProcessor (( span ) => { // Redact sensitive content from inputs span . input . forEach ( msg => { msg . content = msg . content . replace ( / \d {4} [ - ] ? \d {4} [ - ] ? \d {4} [ - ] ? \d {4} / g , '[REDACTED]' ) }) return span })
deregisterProcessor()Deregisters the currently registered processor.
deregisterProcessor (): void
Span kinds Kind Description llmA call to a language model. Use with modelName and modelProvider. embeddingA call to an embedding model. Use with modelName and modelProvider. retrievalA document retrieval operation. toolA tool or function call in an agentic workflow. taskA generic processing task. workflowA multi-step workflow or pipeline. agentAn autonomous agent orchestrating multiple operations.
