MetricContext — evaluation input container

MetricContext is the single input type accepted by every TrustifAI evaluation call. You populate it with a query, the LLM’s answer, and the documents retrieved by your RAG pipeline. All four offline metrics and the async batch pipeline operate on this structure. Embeddings are optional: if you omit them, TrustifAI computes them automatically using the embedding model defined in your config file.

Constructor

from trustifai import MetricContext

context = MetricContext(
    query=query,
    answer=answer,
    documents=documents,
)

str

required

The user’s original question or prompt. Used as the reference point for semantic drift and epistemic consistency scoring.

str

required

The LLM-generated response to evaluate. Evidence coverage and semantic drift are measured against this text.

List[Any]

required

The retrieved context documents passed to the LLM. TrustifAI accepts four formats interchangeably:

LangChain Document — text extracted from .page_content, metadata from .metadata
LlamaIndex NodeWithScore — text extracted from .node.text, metadata from .node.metadata
Plain strings — used as-is, no metadata available
Dicts — text extracted from content, text, or page_content key

np.ndarray

default:"None"

Pre-computed embedding vector for the query. When None, TrustifAI embeds the query on the first evaluation call. Pass pre-computed embeddings to avoid redundant API calls in batch workloads.

np.ndarray

default:"None"

Pre-computed embedding vector for the answer. Used by SemanticDriftMetric and EpistemicConsistencyMetric.

np.ndarray

default:"None"

Pre-computed embedding vectors for each document in documents. Expected shape: (n_docs, embedding_dim). Used by SourceDiversityMetric to compute per-document relevance.

Construction examples

from langchain_core.documents import Document
from trustifai import MetricContext

docs = [
    Document(
        page_content="The Eiffel Tower is located in Paris, France.",
        metadata={"source": "travel-guide.pdf", "page": 12},
    ),
    Document(
        page_content="It was completed in 1889 and stands 330 metres tall.",
        metadata={"source": "travel-guide.pdf", "page": 12},
    ),
]

context = MetricContext(
    query="Where is the Eiffel Tower and how tall is it?",
    answer="The Eiffel Tower is in Paris. It is 330 metres tall and was finished in 1889.",
    documents=docs,
)

from trustifai import MetricContext

context = MetricContext(
    query="What causes thunder?",
    answer="Thunder is caused by the rapid expansion of air heated by lightning.",
    documents=[
        "Lightning superheats surrounding air to roughly 30,000 K.",
        "The rapid expansion of that air creates a shockwave heard as thunder.",
    ],
)

Embeddings are computed on the first get_trust_score call and written back onto the MetricContext object. If you reuse the same context instance for a second call, embeddings will already be populated and no additional embedding API calls are made.

Core API

Data Structures

Metrics

MetricContext — evaluation input container

Constructor

Construction examples

​Constructor

​Construction examples

Constructor

Construction examples