NISIRA Ratings API: Collect Feedback on Bot Responses

The Ratings API lets users leave structured feedback on individual bot messages. Every rating consists of a thumbs-up/thumbs-down value, an optional plain-text comment, and an optional issue tag that categorizes the type of problem encountered. The system enforces a one-rating-per-user-per-message constraint — submitting a second rating for the same message updates the existing record rather than creating a duplicate. Ratings drive the guardrail system and feed into the admin metrics dashboard.

POST /api/ratings/

Submit or update a rating for a bot message. Only "bot" sender messages can be rated — attempting to rate a user message returns a 400 error. If the authenticated user has already rated the target message, the existing record is updated (update_or_create). Use value: "clear" to remove an existing rating.

Request Body

message_id

integer

required

The database ID of the Message record to rate. Obtain this from the message_id field in POST /api/chat/ responses, or from id in the messages array returned by GET /api/conversations/<id>/messages/.

value

string

required

The rating value. Must be one of:

"like" — positive feedback
"dislike" — negative feedback
"clear" — remove an existing rating

comment

string

Optional free-text comment explaining the rating. Maximum 500 characters.

issue_tag

string

default:"none"

Categorical issue tag for dislike ratings. Must be one of the values listed in the Issue Tags section below. Pass "auto" to have the system detect the tag from the comment text using keyword matching.

Response

message_id

integer

ID of the message that was rated.

conversation_id

integer

ID of the conversation containing the rated message.

value

string

The stored rating value ("like" or "dislike"), or null if the rating was cleared.

status

string

Action taken: "created", "updated", "removed", or "noop".

updated_at

string

ISO 8601 timestamp of when the rating was last modified.

issue_tag

string

The issue tag that was stored.

queue_event

object

Details of the async feedback event that was enqueued for downstream processing (telemetry, retraining pipelines, etc.). Only present when a rating was created or updated.

Show Queue event fields

integer

Database ID of the RatingFeedbackEvent record.

status

string

Processing status: "pending", "completed", or "failed".

attempts

integer

Number of processing attempts made.

last_attempt_at

string

ISO 8601 timestamp of the most recent processing attempt.

Example

cURL — Like
cURL — Dislike with tag
Response

curl -X POST https://your-domain.com/api/ratings/ \
  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..." \
  -H "Content-Type: application/json" \
  -d '{
    "message_id": 482,
    "value": "like"
  }'

curl -X POST https://your-domain.com/api/ratings/ \
  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..." \
  -H "Content-Type: application/json" \
  -d '{
    "message_id": 483,
    "value": "dislike",
    "comment": "La respuesta mencionó un reglamento que no existe.",
    "issue_tag": "alucinacion"
  }'

{
  "message_id": 482,
  "conversation_id": 47,
  "value": "like",
  "status": "created",
  "updated_at": "2024-11-10T14:30:00.000Z",
  "issue_tag": "none",
  "queue_event": {
    "id": 91,
    "status": "completed",
    "attempts": 1,
    "last_attempt_at": "2024-11-10T14:30:00.123Z"
  }
}

Issue Tags

Issue tags provide a structured vocabulary for categorizing negative feedback. They are only meaningful on dislike ratings, but can be stored on like ratings as well (with "none" as the default).

Tag value	Label	When to use
`none`	Sin etiqueta	No specific issue — default for all ratings
`irrelevante`	Irrelevante	Response did not address the question asked
`sin_evidencia`	Sin evidencia	Response made claims without citing supporting documents
`tardio`	Tardío	Response took too long to arrive (latency issue)
`alucinacion`	Alucinación	Response contained invented facts, non-existent references, or hallucinated content
`accion_incorrecta`	Acción incorrecta	The recommended action or procedure was wrong
`otro`	Otro	Any other issue not covered by the above categories

You can also pass "auto" as the issue_tag value. The backend will attempt to detect the appropriate tag by scanning the comment field for Spanish-language keywords (e.g., “irrelev”, “sin fuente”, “alucina”, “invent”). If no keyword matches, it falls back to "otro".

GET /api/ratings/summary/

Returns an aggregate statistical summary of all ratings in the system. This endpoint is restricted to staff users (is_staff=True).

Response

total

integer

Total number of ratings across all users and messages.

likes

integer

Total number of positive ratings.

dislikes

integer

Total number of negative ratings.

net_score

integer

Difference between likes and dislikes (likes - dislikes).

satisfaction_rate

float

Proportion of likes to total ratings (0.0–1.0). Used by the guardrail system as a quality threshold.

issue_breakdown

object

Dictionary mapping each issue_tag value to its count across all dislike ratings.

daily_breakdown

array

Per-day rating totals for the last 14 days, each entry with date, total, likes, dislikes.

recent_feedback

array

The 10 most recently updated ratings with full detail: id, message_id, conversation_id, value, comment, issue_tag, updated_at, username.

queue

object

Counts of RatingFeedbackEvent records by status: {pending, failed, processed}.

latest_experiment

object | null

The most recent ExperimentRun record, or null if no experiments exist.

Example

curl -X GET https://your-domain.com/api/ratings/summary/ \
  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."

{
  "total": 214,
  "likes": 178,
  "dislikes": 36,
  "net_score": 142,
  "satisfaction_rate": 0.832,
  "issue_breakdown": {
    "none": 178,
    "alucinacion": 12,
    "irrelevante": 9,
    "sin_evidencia": 8,
    "otro": 7
  },
  "daily_breakdown": [
    { "date": "2024-11-10", "total": 18, "likes": 15, "dislikes": 3 }
  ],
  "recent_feedback": [],
  "queue": { "pending": 0, "failed": 0, "processed": 214 },
  "latest_experiment": null,
  "generated_at": "2024-11-10T17:00:00.000Z",
  "export_url": "https://your-domain.com/api/ratings/export/"
}

GET /api/ratings/export/

Exports all rating records as JSON (default) or CSV. Restricted to staff users (is_staff=True).

Query Parameters

export_format

string

default:"json"

Output format. Must be "json" or "csv".

CSV columns

When export_format=csv, the response is a downloadable file with the following columns: rating_id, message_id, conversation_id, username, value, issue_tag, comment, updated_at.

Example

JSON export
CSV export

curl -X GET "https://your-domain.com/api/ratings/export/?export_format=json" \
  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."

{
  "count": 214,
  "results": [
    {
      "id": 214,
      "message_id": 482,
      "conversation_id": 47,
      "value": "like",
      "comment": "",
      "issue_tag": "none",
      "updated_at": "2024-11-10T14:30:00.000Z"
    }
  ]
}

curl -X GET "https://your-domain.com/api/ratings/export/?export_format=csv" \
  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..." \
  -o ratings_export.csv

Experiments & Guardrails

These endpoints are used to track A/B experiment runs that compare a variant RAG configuration against a baseline, and to check whether the system is operating within quality guardrail thresholds. All require JWT authentication with staff privileges.

GET /api/experiments/

Returns a paginated list of all ExperimentRun records ordered by most recent first.

limit

integer

Maximum number of experiment records to return. Omit to return all.

curl -X GET https://your-domain.com/api/experiments/ \
  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."

GET /api/experiments/latest/

Returns the single most recent ExperimentRun record. Returns 404 if no experiments exist.

curl -X GET https://your-domain.com/api/experiments/latest/ \
  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."

POST /api/experiments/create/

Creates a new experiment run record and automatically evaluates it against configurable guardrail thresholds. If any delta (precision, faithfulness, or latency) breaches its threshold, guardrail_passed is set to false and guardrail_reason is populated. Returns 201 Created on success.

name

string

required

Name of the experiment.

variant_name

string

required

Name of the variant being tested.

executed_by

string

required

Who ran the experiment.

notes

string

Optional free-text notes about the run.

baseline_precision

float

required

Baseline Precision@k score (0–1).

variant_precision

float

required

Variant Precision@k score (0–1).

baseline_faithfulness

float

required

Baseline faithfulness score (0–1).

variant_faithfulness

float

required

Variant faithfulness score (0–1).

baseline_latency

float

required

Baseline average latency in seconds.

variant_latency

float

required

Variant average latency in seconds.

guardrail_threshold_precision

float

default:"-0.01"

Minimum acceptable delta precision. Experiments with lower deltas are blocked.

guardrail_threshold_faithfulness

float

default:"-0.01"

Minimum acceptable delta faithfulness.

guardrail_threshold_latency

float

default:"0.3"

Maximum acceptable latency increase in seconds.

curl -X POST https://your-domain.com/api/experiments/create/ \
  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "top_k_ablation_v2",
    "variant_name": "top_k_10",
    "executed_by": "hugo_dev",
    "baseline_precision": 0.71,
    "variant_precision": 0.74,
    "baseline_faithfulness": 0.83,
    "variant_faithfulness": 0.85,
    "baseline_latency": 2.4,
    "variant_latency": 2.6
  }'

GET /api/guardrails/status/

Evaluates the current system state against all guardrail conditions and returns a pass/fail summary. Checks: the latest experiment’s guardrail_passed flag, any failed feedback events in the queue, and whether satisfaction_rate is above the configured threshold.

satisfaction_threshold

float

default:"0.6"

Minimum acceptable satisfaction rate (0.0–1.0). The guardrail fails if the current rate is below this value.

{
  "guardrail_passed": true,
  "messages": ["Todos los guardrails están en verde."],
  "latest_experiment": null,
  "metrics": {
    "total_feedback": 214,
    "satisfaction_rate": 0.832,
    "net_score": 142,
    "failed_feedback_events": 0
  },
  "generated_at": "2024-11-10T17:00:00.000Z"
}

Overview

Endpoints

NISIRA Ratings API: Collect Feedback on Bot Responses

POST /api/ratings/

Request Body

Response

Example

Issue Tags

GET /api/ratings/summary/

Response

Example

GET /api/ratings/export/

Query Parameters

CSV columns

Example

Experiments & Guardrails

GET /api/experiments/

GET /api/experiments/latest/

POST /api/experiments/create/

GET /api/guardrails/status/

Build docs developers (and LLMs) love

Overview

Endpoints

Documentation Index

​POST /api/ratings/

​Request Body

​Response

​Example

​Issue Tags

​GET /api/ratings/summary/

​Response

​Example

​GET /api/ratings/export/

​Query Parameters

​CSV columns

​Example

​Experiments & Guardrails

​GET /api/experiments/

​GET /api/experiments/latest/

​POST /api/experiments/create/

​GET /api/guardrails/status/

Build docs developers (and LLMs) love

POST /api/ratings/

Request Body

Response

Example

Issue Tags

GET /api/ratings/summary/

Response

Example

GET /api/ratings/export/

Query Parameters

CSV columns

Example

Experiments & Guardrails

GET /api/experiments/

GET /api/experiments/latest/

POST /api/experiments/create/

GET /api/guardrails/status/