Documentation Index
Fetch the complete documentation index at: https://mintlify.com/geremyjampiersalasgarcia-eng/Caso_Practico_Semillero_IA/llms.txt
Use this file to discover all available pages before exploring further.
The ingestion script reads the three Patito S.A. knowledge-base documents from backend/data/raw/, splits them into overlapping chunks, vectorizes each chunk using the Gemini embedding model, and stores the results in three dedicated ChromaDB collections. This script must be run once after the backend environment is set up, and must be re-run whenever any source document changes — the agents cannot answer questions until their collections are populated.
If you start the FastAPI backend before running ingestion, all RAG agents will return errors similar to Collection 'col_catalogo' not found or produce empty responses because there are no vectors to retrieve. Always complete ingestion before directing traffic to the API.
Running ingestion
Activate your Python virtual environment, navigate to the backend/ directory, and run:
cd backend
source .venv/bin/activate # or .venv\Scripts\activate on Windows
python scripts/ingest.py
============================================================
INGESTA DE DOCUMENTOS — Mesa de Ayuda Ventas Patito S.A.
============================================================
Cargando documentos desde: data/raw
Archivo cargado file=01_Catalogo_Productos_Precios.txt chunks=1
Archivo cargado file=02_Politicas_Comerciales_Descuentos_Credito.txt chunks=1
Archivo cargado file=03_Proceso_Ventas_CRM.txt chunks=1
Total de documentos cargados: 3
Limpiando colecciones existentes...
--- Procesando colección: col_catalogo ---
Chunks generados: 42
✅ Colección 'col_catalogo' indexada con 42 chunks
--- Procesando colección: col_politicas ---
Chunks generados: 31
✅ Colección 'col_politicas' indexada con 31 chunks
--- Procesando colección: col_proceso_ventas ---
Chunks generados: 28
✅ Colección 'col_proceso_ventas' indexada con 28 chunks
============================================================
¡INGESTA FINALIZADA EXITOSAMENTE!
============================================================
Resumen de colecciones:
• col_catalogo: 42 documentos indexados
• col_politicas: 31 documentos indexados
• col_proceso_ventas: 28 documentos indexados
GOOGLE_API_KEY must be set in backend/.env before running the ingestion script. Every chunk is vectorized by calling the Gemini Embeddings API (models/gemini-embedding-001), so the script will fail immediately with an authentication error if the key is missing or invalid.
What gets indexed
| Source file | ChromaDB collection | Used by agent(s) |
|---|
01_Catalogo_Productos_Precios.txt | col_catalogo | Catalog & Prices agent, Multimodal Image agent |
02_Politicas_Comerciales_Descuentos_Credito.txt | col_politicas | Commercial Policies agent |
03_Proceso_Ventas_CRM.txt | col_proceso_ventas | Sales Process & CRM agent |
backend/data/raw/ before ingestion is run. The script determines the target collection by matching filename prefixes (01_Catalogo, 02_Politicas, 03_Proceso) — files that do not match any prefix are skipped with a warning.
Ingestion pipeline steps
Load documents from disk
app.rag.loader.load_documents() scans data/raw/ recursively using pathlib. Each .txt file is read by LangChain’s TextLoader with UTF-8 encoding. PDF files are handled by PyPDFLoader. The filename is attached as doc.metadata["filename"] and doc.metadata["source"] for traceability downstream.loader = TextLoader(str(file_path), encoding="utf-8")
docs = loader.load()
Split into overlapping chunks
app.rag.splitter.split_documents() passes the loaded documents through LangChain’s RecursiveCharacterTextSplitter with the following configuration drawn from app.config.settings:splitter = RecursiveCharacterTextSplitter(
chunk_size=settings.CHUNK_SIZE, # default: 1000 characters
chunk_overlap=settings.CHUNK_OVERLAP, # default: 200 characters
separators=["\n\n", "\n", ".", " ", ""]
)
chunks = splitter.split_documents(documents)
\n\n), then line breaks (\n), then sentence endings (.), and finally word boundaries ( ) — preserving semantic coherence in each chunk.Generate embeddings via Gemini
app.rag.vectorstore.get_vectorstore() initializes a Chroma instance with a GoogleGenerativeAIEmbeddings function:from langchain_google_genai import GoogleGenerativeAIEmbeddings
embeddings = GoogleGenerativeAIEmbeddings(
model=settings.EMBEDDING_MODEL_NAME # default: models/gemini-embedding-001
)
Persist vectors to ChromaDB
app.rag.vectorstore.index_documents() calls vectorstore.add_documents(chunks) to write all chunk vectors and metadata into the named ChromaDB collection under data/chroma_db/:vectorstore = Chroma(
collection_name=collection_name, # e.g. "col_catalogo"
embedding_function=embeddings,
persist_directory=settings.CHROMA_PERSIST_DIR # default: data/chroma_db
)
vectorstore.add_documents(documents)
Re-indexing
Because ingest.py clears every collection before re-indexing, it is safe to run the script multiple times. Old vectors are deleted before new ones are inserted, so there is no risk of duplicate or stale chunks accumulating:
# From scripts/ingest.py — clear_collection()
existing = vs.get()
if existing and existing.get("ids"):
vs.delete(ids=existing["ids"])
cd backend
python scripts/ingest.py
API-based ingestion
As an alternative to the command-line script, the FastAPI backend exposes a REST endpoint for triggering ingestion programmatically while the server is running:
POST /api/v1/documents/ingest
{
"directory_path": "data/raw",
"collection_name": "col_catalogo"
}
scripts/ingest.py, this endpoint does not clear the existing collection before inserting new documents. To replace the contents of a collection via the API, you must clear it manually before calling the endpoint.
For the initial setup of a fresh environment, prefer the command-line script — it clears all collections, processes all three source files in one pass, and produces verbose step-by-step log output that makes it easy to spot missing files, Gemini API quota errors, or ChromaDB permission issues before the backend is serving traffic.