Configure Google Gemini AI for Waste Classification

Google Gemini is the intelligence layer at the center of QuipuEco. Every time a user photographs a piece of waste, Gemini’s vision capabilities analyze the image and return a structured JSON payload describing the material type, recycling instructions, estimated CO₂ savings, and the nearest drop-off point. When the user speaks a follow-up question, that same API powers the conversational voice agent, maintaining context across the full dialogue. Both capabilities are served by the same model — gemini-3.1-flash-lite — running in QuipuEco’s FastAPI backend. Before either feature works, you need a Google AI Studio API key.

What Gemini provides to QuipuEco

Vision classification

Analyzes uploaded waste images using multimodal input. Returns a structured JSON object with material type, preparation instructions, CO₂ impact, and the recommended drop-off network.

Conversational chat agent

Powers the voice agent with recycling-specific knowledge. Accepts conversation history and waste context, and returns a structured response that can trigger map navigation or dashboard actions.

gemini-3.1-flash-lite model

A fast, cost-efficient multimodal model optimized for low latency. Suitable for mobile-first experiences where response time directly impacts perceived app quality.

Structured JSON output

Both endpoints request JSON-formatted responses from Gemini using structured prompts defined in quipueco-backend/app/prompts.py, ensuring the frontend can parse results reliably.

Obtaining a Google AI Studio API key

Go to Google AI Studio

Navigate to aistudio.google.com. Sign in with any Google account. Google AI Studio is the official developer console for Gemini API access.

Open the API key panel

Click Get API key in the left sidebar, or navigate directly to aistudio.google.com/apikey.

Create a new API key

Click Create API key. You can either create a key in a new Google Cloud project or associate it with an existing one. For local development, a new project is the easiest option.

Copy the key

The generated key is shown once in full. It always begins with AIza. Copy it immediately and store it somewhere secure (e.g. a password manager) — you will not be able to view it again in full after closing the dialog.

Add the key to your backend .env file

In quipueco-backend/, create or update the .env file:

GEMINI_API_KEY=AIzaSyABCDEFGHIJKLMNOP1234567890example

The FastAPI backend loads this variable at startup via python-dotenv. Restart the backend server after saving the file.

Backend configuration

The API key is loaded by quipueco-backend/app/config.py using python-dotenv and passed to the Gemini SDK client:

quipueco-backend/app/config.py

import os
from dotenv import load_dotenv
import google.generativeai as genai

load_dotenv()

genai.configure(api_key=os.getenv("GEMINI_API_KEY"))
model = genai.GenerativeModel("gemini-3.1-flash-lite")

The model object is then imported by app/main.py and used in both the /clasificar and /chat endpoint handlers.

API endpoints and Gemini integration

POST /clasificar — image classification

The classification endpoint accepts a multipart form upload, encodes the image, and sends it alongside a structured prompt to Gemini’s vision API. The prompt instructs the model to return a specific JSON schema:

quipueco-backend/app/main.py

@app.post("/clasificar")
async def clasificar(file: UploadFile = File(...)):
    image_data = await file.read()
    response = model.generate_content([
        PROMPT_CLASIFICACION,   # from app/prompts.py
        {"mime_type": file.content_type, "data": image_data},
    ])
    return json.loads(response.text)

Classification response schema: The frontend (ClassificationResult.jsx) expects this exact JSON structure from /clasificar:

{
  "nombre": "Botella de plástico PET",
  "tipo": "plastico",
  "co2_evitado_kg": 0.18,
  "peso_estimado_kg": 0.03,
  "instrucciones": "Retira la etiqueta y enjuaga con agua fría antes de entregar.",
  "punto_acopio": "Tiendas Tambo",
  "consejo": "El plástico PET puede reciclarse hasta 10 veces antes de degradarse.",
  "respuesta_voz": "He clasificado una botella de plástico PET. Puedes llevarla a cualquier Tambo cercano."
}

Field	Type	Description
`nombre`	`string`	Human-readable name of the identified item
`tipo`	`string`	Material category: `plastico`, `papel`, `vidrio`, `metal`, `organico`, `electronico`, `peligroso`
`co2_evitado_kg`	`number`	Estimated kg of CO₂ avoided by recycling this item
`peso_estimado_kg`	`number`	Estimated weight of the item in kilograms
`instrucciones`	`string`	Step-by-step preparation instructions before drop-off
`punto_acopio`	`string`	Recommended drop-off network (e.g. “Tiendas Tambo” or “Centro Verde Municipal”)
`consejo`	`string`	Educational tip about the material’s recyclability
`respuesta_voz`	`string`	Pre-formatted text read aloud by the TTS voice agent as the initial greeting

POST /chat — conversational voice agent

The chat endpoint receives the full conversation history, the current waste classification result for context, and the user’s new message. It returns a structured response that optionally triggers a UI action:

quipueco-backend/app/main.py

@app.post("/chat")
async def chat(request: ChatRequest):
    response = model.generate_content([
        PROMPT_CHAT,            # from app/prompts.py
        format_history(request.historial),
        request.mensaje,
    ])
    return json.loads(response.text)

Chat response schema:

{
  "respuesta": "El Tambo más cercano está a 300 metros en Av. Las Alondras 297, Santa Anita.",
  "accion": "mapa",
  "accion_data": "plastico",
  "punto_cercano": "Tambo Alondras - Santa Anita"
}

Field	Type	Description
`respuesta`	`string`	The agent’s reply text, read aloud via the Web Speech API
`accion`	`string \| null`	Optional UI action to trigger: `"mapa"`, `"dashboard"`, or `null`
`accion_data`	`string \| null`	Filter string passed to the map when `accion` is `"mapa"` (e.g. `"plastico"`, `"vidrio"`)
`punto_cercano`	`string \| null`	Name of a specific recycling point for auto-route activation on the map

accion values:

"mapa"

Opens the recycling map, optionally filtered to a specific material category via accion_data.

"dashboard"

Navigates to the Impact Dashboard screen showing the user’s recycling history and points.

null

A text/voice-only response with no navigation action. The agent answers in place.

Request flow diagram

User uploads image or speaks a question
         ↓
   React Frontend
   (ImageCapture.jsx / AgenteVoz.jsx)
         ↓  POST /clasificar or /chat
         ↓  to http://localhost:8000
   FastAPI Backend (app/main.py)
         ↓  image bytes + prompt / history + message
   Google Gemini API
   (gemini-3.1-flash-lite)
         ↓  structured JSON response
   FastAPI parses + returns JSON
         ↓
   Frontend renders result
   TTS reads respuesta_voz / respuesta
   Map navigates if accion = "mapa"

Free tier and rate limits

Google AI Studio provides a free tier suitable for development and hackathon use:

15 requests per minute (RPM) on the free tier for Gemini Flash models
1 million tokens per minute (TPM) input throughput
1,500 requests per day free of charge

These limits are sufficient for local development and live demos. For production deployments with concurrent users, upgrade to a paid tier via Google Cloud Vertex AI to access higher quota tiers and SLA-backed availability.

The free quota in Google AI Studio resets daily at midnight Pacific Time. For a hackathon demo, spacing out classification calls during a presentation ensures you don’t hit the per-minute rate limit during a live session.

Security

Never expose GEMINI_API_KEY in frontend code. Unlike the Mapbox public token — which is designed to be embedded in browser-side JavaScript — the Gemini API key must remain exclusively in quipueco-backend/.env and the server process. Embedding it in the React frontend would make it visible to anyone who inspects the page source or network traffic.If you accidentally commit the key or expose it publicly, revoke it immediately at aistudio.google.com/apikey and generate a new one before continuing.

Get Started

Core Features

Tambo Partnership

Configuration

Configure Google Gemini AI for Waste Classification

What Gemini provides to QuipuEco

Vision classification

Conversational chat agent

gemini-3.1-flash-lite model

Structured JSON output

Obtaining a Google AI Studio API key

Backend configuration

API endpoints and Gemini integration

POST /clasificar — image classification

POST /chat — conversational voice agent

"mapa"

"dashboard"

null

Request flow diagram

Free tier and rate limits

Security

Build docs developers (and LLMs) love

Get Started

Core Features

Tambo Partnership

Configuration

Documentation Index

​What Gemini provides to QuipuEco

Vision classification

Conversational chat agent

gemini-3.1-flash-lite model

Structured JSON output

​Obtaining a Google AI Studio API key

​Backend configuration

​API endpoints and Gemini integration

​POST /clasificar — image classification

​POST /chat — conversational voice agent

"mapa"

"dashboard"

null

​Request flow diagram

​Free tier and rate limits

​Security

Build docs developers (and LLMs) love

What Gemini provides to QuipuEco

Obtaining a Google AI Studio API key

Backend configuration

API endpoints and Gemini integration

POST /clasificar — image classification

POST /chat — conversational voice agent

Request flow diagram

Free tier and rate limits

Security