Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/davidmpizarro/QuipuEco-Hackaton/llms.txt

Use this file to discover all available pages before exploring further.

QuipuEco’s image classification pipeline is the core of the application. When a user photographs or uploads a waste item, the ImageCapture component sends the image to a FastAPI backend that invokes Google Gemini’s vision model. Within seconds the model returns a structured JSON payload that drives the entire result view — from the material category badge and container color to the CO₂ savings estimate, preparation instructions, and the text that the voice agent reads aloud. The resultado.respuesta_voz field is consumed by App.jsx to seed the voice agent’s opening greeting, so users can immediately ask follow-up questions without re-describing the item. The classification is also saved to localStorage before the result view renders, ensuring it is captured in the Impact Dashboard history even if the user navigates away immediately.

​
How Classification Works

1

Select or capture an image

The user picks a file from their device or activates the environment-facing camera. The ImageCapture component accepts JPG, PNG, and WEBP files up to 10 MB. Drag-and-drop is supported on the upload tab.
2

Preview and confirm

After selection the image is rendered in a full-bleed preview panel. A “Clasificar Residuo” button becomes available; a “Volver” button lets the user discard and start over without making a network request.
3

POST /clasificar

Tapping the classify button calls POST /clasificar as multipart/form-data with the file in a field named file. A scanning overlay animation plays while the request is in-flight.
4

Gemini vision analysis

The FastAPI backend forwards the image to Google Gemini’s vision model together with a system prompt that enforces the eight-category taxonomy and the JSON response schema. The model returns a single JSON object.
5

Result rendered

The frontend receives the JSON, calls agregarRegistro(resultado, imagen) to persist the classification to localStorage, then transitions to the ClassificationResult view. App.jsx’s handleResult function immediately invokes SpeechSynthesis with respuesta_voz (falling back to a generic description if the field is absent) — ClassificationResult itself does not call SpeechSynthesis directly.

​
API Endpoint

POST /clasificar
Content-Type: multipart/form-data

file=<binary image data>
The endpoint is configured in ImageCapture.jsx: 
const API_URL = "http://localhost:8000";

const formData = new FormData();
formData.append("file", file);

const { data } = await axios.post(`${API_URL}/clasificar`, formData, {
  headers: { "Content-Type": "multipart/form-data" },
});
API_URL is hardcoded to http://localhost:8000 in ImageCapture.jsx. The FastAPI backend must be running at that address before the PWA is used. Update the constant — or wire it to a VITE_API_URL env variable — before deploying to any non-local environment.

​
Classification Result JSON

The frontend consumes the following fields from the response object. ClassificationResult.jsx reads resultado.nombre, resultado.tipo, resultado.co2_evitado_kg, resultado.peso_estimado_kg, resultado.instrucciones, resultado.punto_acopio, and resultado.consejo directly. The resultado.respuesta_voz field is read by App.jsx to initialise the voice agent greeting. 
{
  "nombre": "Botella de agua PET",
  "tipo": "plastico",
  "co2_evitado_kg": 0.12,
  "peso_estimado_kg": 0.03,
  "instrucciones": "Vacía completamente la botella, retira la tapa y aplástala para reducir volumen.",
  "punto_acopio": "Tiendas Tambo",
  "consejo": "Reutiliza la botella al menos una vez antes de reciclarla para maximizar su vida útil.",
  "respuesta_voz": "He identificado una botella PET reciclable. Recuerda vaciarla y aplastarla antes de llevarla a tu Tambo más cercano."
}
FieldTypeDescription
nombrestringHuman-readable name of the identified object
tipostringMaterial category key — one of the eight supported values
co2_evitado_kgnumberEstimated CO₂ savings in kilograms if the item is recycled correctly
peso_estimado_kgnumberEstimated weight of the item in kilograms
instruccionesstringStep-by-step preparation instructions for the Lima waste system
punto_acopiostringRecommended collection network ("Tiendas Tambo" or "Centro Verde Municipal")
consejostringOne eco-tip shown in the green advice card
respuesta_vozstringText read aloud immediately by App.jsx’s handleResult via SpeechSynthesis when the result arrives — ClassificationResult does not call SpeechSynthesis directly

​
Supported Waste Categories

The ClassificationResult component maps each tipo value to a visual theme via TIPO_CONFIG. All eight categories, their container colors, and display labels are shown below.

🧴 Plástico

Container: Amarillo 🟡
Bottles, bags, packaging, PET containers. Collected at Tambo stores.

📦 Papel / Cartón

Container: Azul 🔵
Cardboard boxes, newspapers, office paper. Collected at Tambo stores.

🍶 Vidrio

Container: Verde 🟢
Bottles, jars, glass containers. Collected at Centro Verde Municipal.

🥬 Orgánico

Container: Naranja 🟠
Food scraps, fruit peels, garden waste. Collected at Centro Verde Municipal.

🥫 Metal

Container: Amarillo 🟡
Cans, aluminium foil, metal lids. Collected at Tambo stores.

📱 Electrónico

Container: Morado 🟣
Phones, batteries, cables, circuit boards. Collected at Centro Verde Municipal.

☣️ Peligroso

Container: Rojo 🔴
Chemicals, paint, solvents, fluorescent bulbs. Collected at Centro Verde Municipal.

🗑️ No Reciclable

Container: Gris ⚫
Contaminated waste that cannot be recovered. Standard trash disposal.

​
Capture Modes

The ImageCapture component exposes two tabs that share the same classification flow after an image is selected.

Upload

Drag-and-drop zone or a “Buscar en dispositivo” file picker button. Accepts image/* (JPG, PNG, WEBP) up to 10 MB. An <input type="file" accept="image/*"> element is hidden and triggered programmatically via a ref.

Camera

Activates the device rear camera via navigator.mediaDevices.getUserMedia({ video: { facingMode: "environment" } }). A live viewfinder renders with a sci-fi scan-line overlay and corner brackets. Tapping “Capturar Foto” draws the current frame to an off-screen canvas and exports it as a JPEG blob at 0.85 quality.
// Camera capture — canvas snapshot
const captureFrame = () => {
  const video = videoRef.current;
  const canvas = document.createElement("canvas");
  canvas.width = video.videoWidth;
  canvas.height = video.videoHeight;
  canvas.getContext("2d").drawImage(video, 0, 0);
  canvas.toBlob((blob) => {
    const f = new File([blob], "captura.jpg", { type: "image/jpeg" });
    setFile(f);
    setPreview(URL.createObjectURL(f));
    stopCamera();
  }, "image/jpeg", 0.85);
};

​
Category Reference

Includes PET bottles, HDPE jugs, plastic bags, polystyrene trays, and blister packaging. Before disposing, rinse clean, remove caps, and flatten to save space. Accepted directly at the Tambo counter — no appointment needed.
Newspapers, magazines, office paper, cardboard boxes, and paper bags. Must be dry and clean — greasy pizza boxes or food-soiled paper cannot be recycled. Flatten boxes before delivery.
Glass bottles, jars, and containers of any color. Remove lids (recycle separately as metal), rinse out, and do not break — broken glass is a safety hazard at collection points. Ceramics and tempered glass are not accepted.
Fruit and vegetable scraps, coffee grounds, eggshells, garden trimmings. Centro Verde processes these for composting. Meat, dairy, and cooked food should be checked with your local center before deposit.
Aluminium cans, tin cans, metal lids, and aluminium foil. Rinse cans, remove labels if possible, and crush to reduce volume. Accepted alongside plastic at Tambo stores.
Mobile phones, laptops, cables, batteries, chargers, and circuit boards. These are RAEE (Residuos de Aparatos Eléctricos y Electrónicos) and require certified handling. Never place them in general trash — they contain heavy metals. Always take to Centro Verde Municipal.
Paints, motor oil, solvents, pesticides, fluorescent tubes, and medical waste. These residues need authorized hazardous-waste handlers. Keep in original sealed containers. Centro Verde Municipal coordinates pickup with certified gestores.
Items that cannot be recovered in Lima’s current infrastructure: dirty multi-layer packaging, contaminated materials, and composite plastics. Dispose of in standard trash. The item still earns 2 eco points for the correct identification.

​
UI State Machine

idle
 └─ [file selected / frame captured]
      └─ preview
           ├─ [Volver] → idle
           └─ [Clasificar Residuo] → loading
                ├─ [API success] → ClassificationResult view
                └─ [API error]  → preview (error banner shown)
If the backend returns an HTTP error, e.response?.data?.detail is shown in the error banner. Common causes are an unsupported file type, a file exceeding the 10 MB limit, or the FastAPI process not running at http://localhost:8000. Verify the backend is healthy before debugging the frontend.
The classification result is saved to localStorage via agregarRegistro() before the result view is rendered. Even if the user navigates away immediately, the entry is preserved in the Impact Dashboard history.

Build docs developers (and LLMs) love

Get started for freeTalk to us