Testing Guide

This guide covers testing MCP servers in guMCP, including writing tests, using test clients, and running the test suite.

Test Structure

All server tests follow a consistent structure:

tests/
├── servers/
│   ├── conftest.py           # Pytest configuration and fixtures
│   ├── test_runner.py        # Test runner script
│   └── your-server/
│       └── tests.py          # Server-specific tests
└── clients/
    ├── LocalMCPTestClient.py  # Local stdio test client
    └── RemoteMCPTestClient.py # Remote SSE test client

Running Tests

Use the test runner from the project root directory:

Local Testing (stdio)

python tests/servers/test_runner.py --server=simple-tools-server

Remote Testing (SSE)

Local SSE Server
Hosted Server

First, start the SSE development server:

./start_sse_dev_server.sh

Then run tests:

python tests/servers/test_runner.py --server=simple-tools-server --remote

Test against a hosted guMCP server:

python tests/servers/test_runner.py \
  --server=simple-tools-server \
  --remote \
  --endpoint=https://mcp.gumloop.com/your-server/{user_id}%3A{api_key}

Writing Tests

Basic Test Structure

Create tests/servers/your-server/tests.py:

import pytest

@pytest.mark.asyncio
async def test_your_feature(client):
    """Test description following the pattern test_<functionality>_<expected_behavior>"""
    # Arrange
    key = "test_key"
    value = "test_value"

    # Act
    response = await client.process_query(
        f"Store the key '{key}' with value '{value}'"
    )

    # Assert
    assert key in response, f"Key '{key}' not found in response: {response}"
    assert value in response, f"Value '{value}' not found in response: {response}"
    print(f"✅ Test passed: {key}={value}")

Using the Client Fixture

The client fixture is automatically provided by conftest.py and gives you access to test client methods:

@pytest.mark.asyncio
async def test_with_client(client):
    # Send natural language query to LLM that uses your MCP tools
    response = await client.process_query("List all stored data")

    # Make assertions about the response
    assert "data" in response.lower()

The test client uses Claude (via Anthropic API) to interact with your MCP server naturally. You need ANTHROPIC_API_KEY in your .env file.

Real-World Test Example

From tests/servers/simple-tools-server/tests.py:

@pytest.mark.asyncio
async def test_store_data(client):
    """Test the store-data tool"""
    key = "test_key"
    value = "test_value"

    response = await client.process_query(
        f"Store the key '{key}' with value '{value}'"
    )

    # Simple verification: check if both key and value appear in response
    assert key in response, f"Key '{key}' not found in response: {response}"
    assert value in response, f"Value '{value}' not found in response: {response}"
    print(f"✅ Data stored successfully: {key}={value}")

@pytest.mark.asyncio
async def test_retrieve_existing_data(client):
    """Test retrieving existing data"""
    key = "test_key"
    expected = "test_value"

    # First store the data
    await client.process_query(
        f"Store the key '{key}' with value '{expected}'"
    )

    # Then retrieve it
    response = await client.process_query(f"Get the value for key '{key}'")

    assert expected in response, \
        f"Expected value '{expected}' not found in response: {response}"
    print(f"✅ Retrieved existing data: {key}={expected}")

@pytest.mark.asyncio
async def test_retrieve_nonexistent_data(client):
    """Test retrieving non-existent data"""
    key = "non_existent_key"

    response = await client.process_query(
        f"Get the value for key '{key}'. "
        f"Respond with 'not found' if it couldn't be found."
    )

    assert "not found" in response.lower(), \
        f"Expected 'not found' message, got: {response}"
    print(f"✅ Correctly handled non-existent key: {key}")

Test Best Practices

Clear Test Names

Use descriptive names following test_<functionality>_<expected_behavior>

Include Integration Tests

Have tests that do NOT use mocks and require actual authentication to verify end-to-end functionality

Helpful Assertions

Include descriptive error messages in assertions to aid debugging

Print Success Messages

Use print statements to show progress when tests pass

Test Organization

import pytest

# Group related tests together

# Create/Write operations
@pytest.mark.asyncio
async def test_create_item(client):
    pass

@pytest.mark.asyncio
async def test_update_item(client):
    pass

# Read operations
@pytest.mark.asyncio
async def test_retrieve_item(client):
    pass

@pytest.mark.asyncio
async def test_list_items(client):
    pass

# Error handling
@pytest.mark.asyncio
async def test_error_missing_params(client):
    pass

Manual Testing with Test Clients

For sandbox/interactive testing, you can use the test clients directly:

Local MCP Test Client

from tests.clients.LocalMCPTestClient import LocalMCPTestClient

client = LocalMCPTestClient()
await client.connect_to_server_by_name("your-server")

# Interactive chat loop starts
# Type queries to test your server

Remote MCP Test Client

from tests.clients.RemoteMCPTestClient import RemoteMCPTestClient

client = RemoteMCPTestClient()
await client.connect_to_server("http://localhost:8000/your-server/local")

# Interactive chat loop starts

Both test clients require ANTHROPIC_API_KEY in your .env file to function.

Test Configuration

The tests/servers/conftest.py provides pytest configuration:

import pytest
import pytest_asyncio

def pytest_addoption(parser):
    """Add command-line options for tests"""
    parser.addoption(
        "--remote",
        action="store_true",
        help="Run tests in remote mode",
    )
    parser.addoption(
        "--endpoint",
        action="store",
        default=None,
        help="URL for the remote server endpoint",
    )

@pytest_asyncio.fixture(scope="function")
async def client(request):
    """Fixture to provide a connected client for all tests"""
    # Automatically provides LocalMCPTestClient or RemoteMCPTestClient
    # based on --remote flag

pytest Options

The test runner uses these pytest options:

-v - Verbose output
--capture=no - Show print statements
-p no:warnings - Disable warning capture
--import-mode=importlib - Use importlib for imports

Debugging Tests

View Detailed Output

python tests/servers/test_runner.py --server=your-server -v

Run Specific Tests

python tests/servers/test_runner.py \
  --server=your-server \
  -k test_specific_function

Stop on First Failure

python tests/servers/test_runner.py --server=your-server -x

Testing Requirements

From requirements-dev.txt:

pytest>=7.4.0
pytest-asyncio>=0.21.1
pytest-timeout
pytest-mock==3.11.1

Writing Test Assertions

Good Practices

# Good: Descriptive error messages
assert key in response, f"Key '{key}' not found in response: {response}"

# Good: Clear expectations
assert response.status_code == 200, f"Expected 200, got {response.status_code}"

# Good: Check specific conditions
assert "error" not in response.lower(), "Unexpected error in response"

Avoid

# Bad: No error message
assert key in response

# Bad: Vague assertion
assert response

# Bad: Testing too many things at once
assert key in response and value in response and "success" in response

Pre-Submission Checklist

Before submitting a PR, ensure:

All tests pass

python tests/servers/test_runner.py --server=your-server

Tests cover new functionality

Every new tool, resource, or prompt should have corresponding tests

Include integration tests

At least some tests should run without mocks and require actual authentication

Tests are documented

Each test has a clear docstring explaining what it tests

Next Steps

Review Code Style for code quality standards
See Creating Servers for implementation patterns
Read Contributing Guidelines before submitting

Overview

Getting Started

Core Concepts

Deployment

Development

Testing Guide

Test Structure

Running Tests

Local Testing (stdio)

Remote Testing (SSE)

Writing Tests

Basic Test Structure

Using the Client Fixture

Real-World Test Example

Test Best Practices

Clear Test Names

Include Integration Tests

Helpful Assertions

Print Success Messages

Test Organization

Manual Testing with Test Clients

Local MCP Test Client

Remote MCP Test Client

Test Configuration

pytest Options

Debugging Tests

View Detailed Output

Run Specific Tests

Stop on First Failure

Testing Requirements

Writing Test Assertions

Good Practices

Avoid

Pre-Submission Checklist

Next Steps

Build docs developers (and LLMs) love

Overview

Getting Started

Core Concepts

Deployment

Development

Documentation Index

​Test Structure

​Running Tests

​Local Testing (stdio)

​Remote Testing (SSE)

​Writing Tests

​Basic Test Structure

​Using the Client Fixture

​Real-World Test Example

​Test Best Practices

Clear Test Names

Include Integration Tests

Helpful Assertions

Print Success Messages

​Test Organization

​Manual Testing with Test Clients

​Local MCP Test Client

​Remote MCP Test Client

​Test Configuration

​pytest Options

​Debugging Tests

​View Detailed Output

​Run Specific Tests

​Stop on First Failure

​Testing Requirements

​Writing Test Assertions

​Good Practices

​Avoid

​Pre-Submission Checklist

​Next Steps

Build docs developers (and LLMs) love

Test Structure

Running Tests

Local Testing (stdio)

Remote Testing (SSE)

Writing Tests

Basic Test Structure

Using the Client Fixture

Real-World Test Example

Test Best Practices

Test Organization

Manual Testing with Test Clients

Local MCP Test Client

Remote MCP Test Client

Test Configuration

pytest Options

Debugging Tests

View Detailed Output

Run Specific Tests

Stop on First Failure

Testing Requirements

Writing Test Assertions

Good Practices

Avoid

Pre-Submission Checklist

Next Steps