Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/mastra-ai/mastra/llms.txt

Use this file to discover all available pages before exploring further.

Mastra’s tracing system captures complete execution paths through your agents and workflows, providing visibility into every step of processing.

​
What is Tracing?

Tracing creates a hierarchical record (trace) of all operations in a request:
  • Trace - Complete execution of a request (identified by traceId)
  • Span - Individual operation within a trace (agent call, tool execution, etc.)
  • Parent-Child Relationships - Spans nest to show execution flow

​
Basic Usage

​
Automatic Tracing

All agent and workflow operations are automatically traced: 
import { Mastra } from '@mastra/core';

const mastra = new Mastra({
  observability: { enabled: true },
  agents: {
    myAgent: {
      name: 'My Agent',
      instructions: 'You are helpful',
      model: 'gpt-4',
    },
  },
});

// This creates a complete trace automatically
const result = await mastra.getAgent('myAgent').generate({
  messages: [{ role: 'user', content: 'What is 2+2?' }],
});

console.log('Trace ID:', result.traceId);
// Trace ID: a1b2c3d4e5f67890a1b2c3d4e5f67890

​
Retrieving Traces

Access trace data after execution: 
const result = await agent.generate({
  messages: [{ role: 'user', content: 'Hello' }],
});

// Get the complete trace
const trace = await mastra.getTrace(result.traceId!);

console.log('Root span:', trace.rootSpan);
console.log('All spans:', trace.spans);
console.log('Span count:', trace.spans.length);

​
Span Types

​
Agent Run Span

Root span for agent execution: 
const result = await agent.generate({
  messages: [{ role: 'user', content: 'Hello' }],
});

const trace = await mastra.getTrace(result.traceId!);
const agentSpan = trace.rootSpan;

console.log(agentSpan);
// {
//   type: 'agent_run',
//   name: 'myAgent',
//   entityType: 'agent',
//   entityId: 'myAgent',
//   attributes: {
//     conversationId: 'thread-123',
//     instructions: 'You are helpful',
//     availableTools: ['calculator'],
//     maxSteps: 5
//   },
//   startTime: Date(...),
//   endTime: Date(...),
//   input: [...messages],
//   output: [...response]
// }

​
Model Generation Span

Captures LLM API calls: 
// Model generation span attributes
{
  type: 'model_generation',
  attributes: {
    model: 'gpt-4',
    provider: 'openai',
    
    // Token usage
    usage: {
      inputTokens: 150,
      outputTokens: 75,
      inputDetails: {
        text: 100,
        cacheRead: 50    // Anthropic cache hits
      },
      outputDetails: {
        text: 60,
        reasoning: 15     // o1 reasoning tokens
      }
    },
    
    // Model parameters
    parameters: {
      temperature: 0.7,
      maxOutputTokens: 1000,
      topP: 0.9
    },
    
    streaming: true,
    finishReason: 'stop',
    responseModel: 'gpt-4-0613',
    responseId: 'chatcmpl-123'
  }
}

​
Tool Call Span

Captures tool/function execution: 
// Tool call span
{
  type: 'tool_call',
  name: 'calculator',
  attributes: {
    toolType: 'function',
    toolDescription: 'Perform calculations',
    success: true
  },
  input: { operation: 'add', a: 2, b: 2 },
  output: { result: 4 }
}

​
Workflow Spans

Captures workflow execution: 
// Workflow run span
{
  type: 'workflow_run',
  name: 'myWorkflow',
  attributes: {
    status: 'completed'
  }
}

// Workflow step span
{
  type: 'workflow_step',
  name: 'processData',
  attributes: {
    status: 'completed'
  },
  input: { data: [...] },
  output: { processed: [...] }
}

​
Trace Metadata

​
Adding Metadata

Attach custom metadata to traces: 
const result = await agent.generate({
  messages: [{ role: 'user', content: 'Hello' }],
  
  tracingOptions: {
    metadata: {
      userId: 'user-123',
      source: 'web-app',
      version: '1.0.0',
      experimentId: 'exp-456',
    },
  },
});

// Metadata is stored in root span
const trace = await mastra.getTrace(result.traceId!);
console.log(trace.rootSpan.metadata);
// { userId: 'user-123', source: 'web-app', ... }

​
Using Tags

Categorize traces with tags: 
const result = await agent.generate({
  messages: [{ role: 'user', content: 'Hello' }],
  
  tracingOptions: {
    tags: ['production', 'user-query', 'priority-high'],
  },
});

// Query traces by tags (implementation depends on platform)
const traces = await observability.queryTraces({
  tags: ['production'],
});

​
Request Context Keys

Automatically extract fields from RequestContext: 
import { RequestContext } from '@mastra/core/request-context';

const requestContext = new RequestContext();
requestContext.set('userId', 'user-123');
requestContext.set('session.experimentId', 'exp-456');
requestContext.set('user.tier', 'premium');

const result = await agent.generate({
  messages: [{ role: 'user', content: 'Hello' }],
  requestContext,
  
  tracingOptions: {
    // Extract these keys as metadata (supports dot notation)
    requestContextKeys: ['userId', 'session.experimentId', 'user.tier'],
  },
});

// Metadata includes extracted context
const trace = await mastra.getTrace(result.traceId!);
console.log(trace.rootSpan.metadata);
// { userId: 'user-123', session.experimentId: 'exp-456', user.tier: 'premium' }

​
Privacy Controls

​
Hiding Input/Output

Protect sensitive data: 
const result = await agent.generate({
  messages: [{ role: 'user', content: 'My SSN is 123-45-6789' }],
  
  tracingOptions: {
    hideInput: true,   // Input won't be stored in spans
    hideOutput: true,  // Output won't be stored in spans
  },
});

​
Redacting Stream Data

Control what’s sent to clients in streams: 
import { MastraServer } from '@mastra/hono';

const server = new MastraServer({
  app,
  mastra,
  streamOptions: {
    redact: true,  // Default: redacts system prompts, tool definitions, API keys
  },
});

​
Trace Linking

​
Parent Trace ID

Link traces across service boundaries: 
// Service A creates a trace
const resultA = await agentA.generate({
  messages: [{ role: 'user', content: 'Start' }],
});

const traceId = resultA.traceId!;

// Service B continues the trace
const resultB = await agentB.generate({
  messages: [{ role: 'user', content: 'Continue' }],
  
  tracingOptions: {
    traceId,  // Continue existing trace
  },
});

// Both operations appear in same trace
console.log(resultA.traceId === resultB.traceId); // true

​
Parent Span ID

Create child spans in existing trace: 
const resultA = await agentA.generate({
  messages: [{ role: 'user', content: 'Start' }],
});

const traceA = await mastra.getTrace(resultA.traceId!);
const parentSpanId = traceA.rootSpan.id;

// Create child span
const resultB = await agentB.generate({
  messages: [{ role: 'user', content: 'Continue' }],
  
  tracingOptions: {
    traceId: resultA.traceId!,
    parentSpanId,
  },
});

​
Annotations

​
Adding Scores

Add quality scores to traces: 
const result = await agent.generate({
  messages: [{ role: 'user', content: 'Hello' }],
});

const trace = await mastra.getTrace(result.traceId!);

// Add score to trace
trace.addScore({
  name: 'accuracy',
  value: 0.95,
  comment: 'High accuracy response',
});

// Add score to specific span
const toolSpan = trace.spans.find(s => s.type === 'tool_call');
toolSpan?.addScore({
  name: 'relevance',
  value: 0.8,
});

​
Adding Feedback

Capture user feedback: 
const trace = await mastra.getTrace(traceId);

trace.addFeedback({
  value: 1,  // thumbs up
  comment: 'Great response!',
});

// Or thumbs down
trace.addFeedback({
  value: -1,
  comment: 'Incorrect information',
});

​
Span Navigation

​
Tree Traversal

Navigate span hierarchy: 
const trace = await mastra.getTrace(traceId);

// Root span
const root = trace.rootSpan;
console.log(root.name);

// Child spans
root.children.forEach(child => {
  console.log(`  ${child.name}`);
  
  // Grandchildren
  child.children.forEach(grandchild => {
    console.log(`    ${grandchild.name}`);
  });
});

​
Finding Spans

Locate specific spans: 
const trace = await mastra.getTrace(traceId);

// Find all tool call spans
const toolSpans = trace.spans.filter(s => s.type === 'tool_call');

// Find specific span by ID
const span = trace.getSpan('span-id-123');

// Find first model generation
const modelSpan = trace.spans.find(s => s.type === 'model_generation');

​
Performance Metrics

Calculate performance metrics from spans: 
const trace = await mastra.getTrace(traceId);

// Total duration
const duration = trace.rootSpan.endTime! - trace.rootSpan.startTime;
console.log(`Total duration: ${duration}ms`);

// Token usage across all model calls
const modelSpans = trace.spans.filter(s => s.type === 'model_generation');
const totalTokens = modelSpans.reduce((sum, span) => {
  const usage = span.attributes?.usage;
  return sum + (usage?.inputTokens || 0) + (usage?.outputTokens || 0);
}, 0);
console.log(`Total tokens: ${totalTokens}`);

​
Next Steps

Logging

Configure structured logging

Evals

Evaluate agent quality with scorers

Build docs developers (and LLMs) love

Get started for freeTalk to us