Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/JoAmps/rgt-assignment/llms.txt

Use this file to discover all available pages before exploring further.

The RAG Support System includes comprehensive unit tests that validate functionality without requiring external API calls. All tests mock external services to run offline.

​
Overview

The test suite covers:
  • RAG retrieval and answer generation
  • Document ingestion pipelines
  • ML model training and prediction
  • API endpoints and services
  • Structured outputs and verification

​
Prerequisites

  • Python 3.12+ installed
  • Dependencies installed via uv sync
  • Test files located in tests/ directory

​
Running Tests

​
Full Test Suite (Quiet Mode)

Run all tests with minimal output: 
pytest -q
Output: 
......................                                    [100%]
22 passed in 3.45s

​
Verbose Mode with Print Statements

Run tests with detailed output and capture print statements: 
uv run pytest -s -v tests/
Output: 
tests/test_retriever.py::test_search_and_answer PASSED
tests/test_ingestion.py::test_ingest_document PASSED
tests/test_ml_training.py::test_train_models PASSED
...
Flags:
  • -s: Show print statements and logging output
  • -v: Verbose mode (show test names)

​
Run Specific Test File

Execute tests from a single file: 
pytest tests/test_retriever.py -q

​
Run Specific Test Function

Execute a single test: 
pytest tests/test_retriever.py::test_search_and_answer -q

​
Test Structure

Tests are organized in the tests/ directory: 
tests/
├── test_retriever.py       # RAG retrieval tests
├── test_ingestion.py       # Document ingestion tests
├── test_ml_training.py     # Model training tests
├── test_ml_predict.py      # Prediction tests
├── test_api.py             # API endpoint tests
└── conftest.py             # Shared fixtures and mocks

​
Mocking External Services

All tests mock external API calls to ensure offline execution:

​
Mocked Services

  • OpenAI API: LLM calls and embeddings
  • Chroma: Vector database operations
  • Unstructured API: Document parsing
Example mock usage: 
import pytest
from unittest.mock import Mock, patch

@patch("src.rag.retriever.OpenAIEmbeddings")
def test_retrieval(mock_embeddings):
    mock_embeddings.return_value.embed_query.return_value = [0.1] * 1536
    # Test retrieval logic
    assert result is not None

​
Test Coverage

​
RAG Retrieval Tests

Validate search and answer generation: 
def test_search_and_answer():
    """Test that RAG retrieval returns relevant documents and answers."""
    # Mock Chroma search
    # Mock LLM generation
    # Assert correct documents retrieved
    # Assert answer quality

​
Ingestion Tests

Validate document processing: 
def test_ingest_document():
    """Test document loading, chunking, and storage."""
    # Mock Unstructured loader
    # Test chunking logic
    # Assert correct metadata

​
ML Training Tests

Validate model training: 
def test_train_models():
    """Test category and priority model training."""
    # Provide sample training data
    # Assert models are trained
    # Assert metrics are computed

​
API Tests

Validate HTTP endpoints: 
def test_ingest_endpoint():
    """Test POST /ingest endpoint."""
    response = client.post("/ingest", json={"filepath": "/path/to/doc.md"})
    assert response.status_code == 200

​
Running Tests in CI

The project includes a GitHub Actions workflow (.github/workflows/ci.yml) that:
  1. Sets up Python environment
  2. Installs dependencies
  3. Runs full test suite
  4. Reports test results

​
Test Fixtures

Common fixtures are defined in conftest.py: 
import pytest

@pytest.fixture
def sample_ticket():
    """Provide sample ticket data for tests."""
    return {
        "subject": "Billing issue",
        "body": "I was charged twice",
        "user_question": "How do I get a refund?"
    }

@pytest.fixture
def mock_chroma_client():
    """Mock Chroma vector database client."""
    # Return mocked Chroma client

​
Test Output and Reports

Pytest generates detailed output:

​
Success Output

================================ test session starts =================================
platform linux -- Python 3.12.0, pytest-7.4.3, pluggy-1.3.0
rootdir: /path/to/project
collected 22 items

tests/test_retriever.py::test_search_and_answer PASSED                      [  4%]
tests/test_ingestion.py::test_ingest_document PASSED                        [  9%]
tests/test_ml_training.py::test_train_models PASSED                         [ 13%]
...

================================ 22 passed in 3.45s ==================================

​
Failure Output

================================== FAILURES ==========================================
_________________________ test_search_and_answer _________________________________

    def test_search_and_answer():
>       assert result is not None
E       AssertionError: assert None is not None

tests/test_retriever.py:42: AssertionError
================================ 1 failed, 21 passed in 3.52s ========================

​
Best Practices

​
Running Tests During Development

  1. Before committing: Run full test suite 
    pytest -q
  2. During feature development: Run relevant test file 
    pytest tests/test_feature.py -v
  3. Debugging failures: Use verbose mode with prints 
    pytest tests/test_feature.py::test_specific -s -v

​
Writing New Tests

When adding new functionality:
  1. Create test file in tests/ directory
  2. Import necessary mocks from conftest.py
  3. Write test functions with test_ prefix
  4. Mock all external API calls
  5. Assert expected behavior
Example: 
import pytest
from unittest.mock import patch
from src.my_module import my_function

@patch("src.my_module.external_service")
def test_my_function(mock_service):
    mock_service.return_value = "mocked_response"
    result = my_function()
    assert result == "expected_value"

​
Troubleshooting

Ensure you’re running pytest from the project root directory and dependencies are installed:
uv sync
Tests should NOT require real API keys. If you see this error, mocking may be incomplete. Check that:
  • External services are properly mocked
  • Environment variables are not being accessed in test code
Common causes:
  • Missing test dependencies in CI environment
  • Path differences between local and CI
  • Race conditions in parallel test execution
This usually indicates:
  • Circular import in source code
  • Missing __init__.py files
  • Incorrect import paths

​
Additional Commands

​
Run tests with coverage report

pytest --cov=src --cov-report=html tests/

​
Run tests in parallel (requires pytest-xdist)

pytest -n auto tests/

​
Run only tests marked with specific marker

pytest -m "integration" tests/

Build docs developers (and LLMs) love

Get started for freeTalk to us