Mindloom Inference Service: Endpoints and Lifecycle

The Mindloom inference service is a stateless FastAPI microservice responsible for one job: running the ML model. It classifies content, extracts keywords, produces embeddings, assigns cluster IDs, and calculates UMAP coordinates — all in a single /predict call. It is an internal service, called only by the api/ layer, and is never exposed directly to the web client or to the public internet.

Startup Lifecycle

Before the service can respond to any request, it must load two assets from different sources. This split between build time and runtime is intentional and critical to understand.

Docker build: bake the transformer

During docker build, the intfloat/multilingual-e5-small transformer (~470 MB) is downloaded from HuggingFace and stored in the image layer at HF_HOME=/app/.cache/huggingface. The container can start offline — no HuggingFace download happens at runtime.

ENV HF_HOME=/app/.cache/huggingface
RUN python -c "from sentence_transformers import SentenceTransformer; \
    SentenceTransformer('intfloat/multilingual-e5-small')"

Container startup: download model.joblib

When the container starts, the FastAPI lifespan handler calls load_model(), which downloads models/latest.txt from OCI Object Storage to find the active version prefix, then downloads the referenced model.joblib (~0.24 MB).

Populate APP_STATE

The loaded artifact dict is stored in APP_STATE["model"]. A SentenceTransformer is instantiated from meta["embedding_model"] — this reads from the local HuggingFace cache baked in during build, not from the network. The service sets APP_STATE["model_loaded"] = True only after both are ready.

Healthcheck passes

The Docker healthcheck polls GET /health every 30 seconds with a 60-second start period. Once the service responds with "status": "ok", it transitions to healthy and the api/ container is allowed to start (configured with depends_on: condition: service_healthy).

If load_model() fails for any reason — missing bucket credentials, a non-existent latest.txt, or a mismatched artifact — the service will never become healthy. Because the api/ container has a service_healthy dependency on inference, the entire application stack will not start. A broken model deployment is a full outage.

APP_STATE Keys

The global APP_STATE dict is the runtime state store for the process:

Key	Type	Contents
`model_loaded`	`bool`	`True` only after startup completes successfully
`model`	`dict`	The full `model.joblib` artifact dict
`encoder`	`SentenceTransformer`	The loaded `intfloat/multilingual-e5-small` encoder
`meta`	`dict`	`artifact["meta"]` — version, dims, prefixes, categories, metrics
`kmeans`	`KMeans`	`artifact.get("kmeans")` — for cluster assignment
`umap`	`UMAP`	`artifact.get("umap_reducer")` — for 2D projection

OCI Authentication

The oci_client.py module selects its authentication strategy based on the environment:

def _bucket_client():
    try:
        # On the VM: Instance Principal — no explicit credentials needed
        signer = oci.auth.signers.InstancePrincipalsSecurityTokenSigner()
        return oci.object_storage.ObjectStorageClient({}, signer=signer)
    except (oci.exceptions.ConfigFileNotFound, oci.exceptions.InvalidConfig):
        # Locally: use ~/.oci/config (API key)
        return oci.object_storage.ObjectStorageClient(oci.config.from_file())

On the production VM, no credentials file is needed — the OCI SDK obtains a short-lived token via the Instance Metadata Service. In local development, the same code falls back to ~/.oci/config.

For local development without OCI credentials, set the MODEL_LOCAL_PATH environment variable to the path of a local model.joblib file. When this variable points to an existing file, load_model() skips OCI entirely and loads the artifact directly — no ~/.oci/config required.

Environment Variables

Variable	Required	Purpose
`MODEL_BUCKET`	In production	OCI Object Storage bucket name containing `model.joblib`
`MODEL_LOCAL_PATH`	No	Dev shortcut — loads a local file and skips OCI entirely

API Endpoints

POST /predict

Classifies a text document and returns the full set of fields needed to index the content in the database — category, probability, keywords, explainability terms, embedding vector, cluster ID, and map coordinates — all in a single call.

Request
Response

{
  "text": "En este contenido se presentan los conceptos básicos de Spring Boot."
}

Field	Type	Required
`text`	string	Yes

{
  "category": "Backend",
  "probability": 0.89,
  "keywords": ["java", "spring", "rest", "boot", "endpoint"],
  "explanation": ["spring", "rest", "endpoint", "java", "controller"],
  "embedding": [0.021, -0.118, "…384 floats…"],
  "cluster_id": 3,
  "x": 4.21,
  "y": -1.07
}

Field	Type	Description
`category`	string	One of the 8 model categories
`probability`	float 0–1	Confidence score for the predicted class
`keywords`	string[]	Top 5 TF-IDF terms with non-zero weight
`explanation`	string[]	Top 5 terms weighted by classifier coefficients
`embedding`	float32[384]	L2-normalized vector for the `VECTOR(384)` database column
`cluster_id`	int	K-means cluster assignment
`x`, `y`	float	UMAP 2D coordinates for the corpus map

This endpoint always applies the "passage: " E5 prefix internally. The returned embedding is the raw 384-dim vector — not the 884-dim classifier feature vector. This is correct: the database stores and searches over the 384-dim space.

POST /embed

Returns a single L2-normalized embedding vector. Used by the api/ layer’s semantic search path, which then passes the vector to the database for VECTOR_DISTANCE ranking.

Request
Response

{
  "text": "cómo validar entradas en Spring",
  "type": "query"
}

Field	Type	Required	Description
`text`	string	Yes	Text to encode
`type`	`"query"` \| `"passage"`	Yes	E5 prefix to apply

{
  "embedding": [0.021, -0.118, "…384 floats…"]
}

Returns exactly 384 L2-normalized float values.

The type field is a Pydantic Literal["query", "passage"] — any other value returns 422 Unprocessable Entity immediately. There is no default value by design.

Passing type="passage" to /embed when performing a search (instead of type="query") returns a vector that is semantically offset from what the database expects. Search results will be silently degraded — no error is raised and the response status will be 200 OK. Always use type="query" for search and type="passage" for content indexing.

GET /health

Returns the service readiness status. This is the endpoint polled by the Docker healthcheck.

{
  "status": "ok",
  "model_loaded": true,
  "version": "v1"
}

Field	Values	Description
`status`	`"ok"` \| `"error"`	Reflects whether the model is loaded
`model_loaded`	`true` \| `false`	`true` only after startup completes
`version`	`"v1"`	Fixed service version string

The Docker healthcheck configuration in the Dockerfile uses urllib (not curl, which is absent from the slim image):

HEALTHCHECK --interval=30s --timeout=5s --start-period=60s --retries=3 \
  CMD python -c "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8000/health').status==200 else 1)"

GET /model/info

Returns the full meta block from the loaded artifact. Returns 503 if the model has not finished loading.

{
  "version": "v1",
  "embedding_model": "intfloat/multilingual-e5-small",
  "dim": 384,
  "feature_dim": 884,
  "svd_components": 500,
  "classifier_c": 4.0,
  "doc_prefix": "passage: ",
  "query_prefix": "query: ",
  "categories": ["Backend", "Frontend", "Móvil", "Datos e IA", "DevOps y Cloud", "Bases de datos", "Seguridad", "Fundamentos"],
  "n_clusters": 8,
  "terms_by_category": { "Backend": ["spring", "java", "..."] },
  "metrics": {
    "embedding_macro_f1_en": 0.0,
    "embedding_macro_f1_es": 0.0,
    "tfidf_macro_f1_en": 0.0,
    "tfidf_macro_f1_es": 0.0,
    "embedding_macro_f1_es_reliable": 0.0,
    "es_reliable_categories": 8,
    "es_min_support": 30
  },
  "train_size": 0
}

dim and feature_dim are two different numbers. dim (384) is the size of the embedding vector stored in the database and returned in API responses. feature_dim (884) is what the classifier consumes internally. See Model Overview for the full explanation.

Network Topology

The inference service port (8000) is not published to the host. The service is only reachable from within the internal Docker Compose network (techmind). Only the api/ container can call it. Exposing it externally would bypass all input validation in the API layer.

Troubleshooting

Container restarts in a loop

load_model() raised an exception during the FastAPI lifespan. Common causes:

Bucket credentials missing: on a new VM, ensure the Instance Principal policy is configured in OCI IAM.
models/latest.txt does not exist: the model artifact has not been uploaded yet, or the bucket name in MODEL_BUCKET is incorrect.
latest.txt points to a version that doesn’t exist: the pointer was updated before the artifact finished uploading. Re-upload the full version and verify the object exists in the bucket before updating the pointer.

Container stays 'unhealthy' after 60 seconds

load_model() is taking longer than the start-period allows. This can happen when the bucket is geographically distant or under load. Increase the --start-period value in the HEALTHCHECK directive in the Dockerfile and rebuild the image.

KeyError on startup

The model.joblib artifact’s keys do not match what the inference code expects. This happens when a key is renamed or added in the training notebook without updating the inference service. Ensure the artifact was produced by the current version of TechMind_02_Model_Training.ipynb and that all expected keys are present. See Model Overview for the full list of required keys.

Docker image is 4× the expected size

Torch was installed from the default PyPI index on an x86 machine, which resolves to the CUDA build and pulls ~2 GB of unused GPU libraries. Install torch explicitly from the CPU wheel index:

pip install torch --index-url https://download.pytorch.org/whl/cpu

This is already the default in the Dockerfile. The issue only appears when building manually on x86 without following the Dockerfile steps. On aarch64 (the production VM), no CUDA wheels exist in the default index, so this is not a concern in production.

Get Started

Core Features

Deployment

Data & ML

Mindloom Inference Service: Endpoints and Lifecycle

Startup Lifecycle

APP_STATE Keys

OCI Authentication

Environment Variables

API Endpoints

POST /predict

POST /embed

GET /health

GET /model/info

Network Topology

Troubleshooting

Build docs developers (and LLMs) love

Get Started

Core Features

Deployment

Data & ML

Documentation Index

​Startup Lifecycle

​APP_STATE Keys

​OCI Authentication

​Environment Variables

​API Endpoints

​POST /predict

​POST /embed

​GET /health

​GET /model/info

​Network Topology

​Troubleshooting

Build docs developers (and LLMs) love

Startup Lifecycle

APP_STATE Keys

OCI Authentication

Environment Variables

API Endpoints

POST /predict

POST /embed

GET /health

GET /model/info

Network Topology

Troubleshooting