Testing

Evaly uses a comprehensive testing strategy combining unit tests and end-to-end (E2E) tests to ensure reliability and quality.

Testing Stack

Evaly uses two primary testing frameworks:

Vitest: Fast unit testing framework for components and utilities
Playwright: End-to-end testing for complete user workflows
React Testing Library: Component testing utilities

Unit Testing with Vitest

Vitest is used for testing individual components, hooks, and utility functions in isolation.

Running Unit Tests

# Run all unit tests once
bun run test

Writing Unit Tests

While the testing infrastructure is configured, unit test files don’t currently exist in the codebase. When writing unit tests, follow these patterns: Test File Naming:

Place test files next to the code they test
Use .test.tsx or .spec.tsx extension
Example: Button.tsx → Button.test.tsx

Example Component Test:

import { render, screen } from '@testing-library/react';
import { describe, it, expect } from 'vitest';
import { Button } from './Button';

describe('Button', () => {
  it('renders with correct text', () => {
    render(<Button>Click me</Button>);
    expect(screen.getByText('Click me')).toBeInTheDocument();
  });
  
  it('calls onClick when clicked', async () => {
    const handleClick = vi.fn();
    render(<Button onClick={handleClick}>Click me</Button>);
    
    await userEvent.click(screen.getByText('Click me'));
    expect(handleClick).toHaveBeenCalledOnce();
  });
});

Example Utility Test:

import { describe, it, expect } from 'vitest';
import { formatScore } from './score-utils';

describe('formatScore', () => {
  it('formats score as percentage', () => {
    expect(formatScore(85, 100)).toBe('85%');
  });
  
  it('handles zero correctly', () => {
    expect(formatScore(0, 100)).toBe('0%');
  });
});

Unit tests are currently not implemented in the codebase. The testing infrastructure is configured and ready to use when needed.

End-to-End Testing with Playwright

Playwright tests simulate real user interactions and test complete workflows across the application.

Running E2E Tests

# Run all end-to-end tests
bun run test:e2e

E2E Test Configuration

The Playwright configuration is located in playwright.config.ts with the following key settings:

{
  testDir: './e2e',
  fullyParallel: false,  // Required due to Convex Auth refresh token rotation
  workers: 1,            // Single worker to prevent auth conflicts
  timeout: 60000,        // 60 second timeout for SSR apps
  retries: process.env.CI ? 2 : 1,  // Retry flaky tests
  
  webServer: {
    command: 'bun run dev',
    url: 'http://localhost:3000',
    reuseExistingServer: !process.env.CI,
    timeout: 120000,
  }
}

Important: E2E tests run with a single worker (workers: 1) to prevent refresh token rotation conflicts. Convex Auth uses refresh token rotation, which means parallel test execution can cause authentication failures.

E2E Test Structure

End-to-end tests are organized by user role and feature:

e2e/
├── tests/
│   ├── auth.spec.ts                    # Authentication flows
│   ├── organizer/                      # Organizer features
│   │   ├── test-crud.spec.ts          # Test creation and management
│   │   ├── questions.spec.ts          # Question management
│   │   ├── question-types.spec.ts     # Different question types
│   │   ├── question-library.spec.ts   # Question bank features
│   │   ├── grading.spec.ts            # Manual grading workflow
│   │   ├── results.spec.ts            # Test results and analytics
│   │   ├── publishing.spec.ts         # Publishing tests
│   │   ├── access-control.spec.ts     # Access restrictions
│   │   ├── organization.spec.ts       # Organization management
│   │   └── settings.spec.ts           # Settings management
│   ├── participant/                    # Participant features
│   │   └── access-control.spec.ts     # Participant access controls
│   └── integration/                    # Full user journeys
│       └── full-flow.spec.ts          # Complete test lifecycle
├── fixtures/
│   └── test-helpers.ts                # Reusable test utilities
├── auth.setup.ts                       # Authentication setup
└── global-setup.ts                     # Global test setup

Writing E2E Tests

Example Organizer Test:

import { test, expect } from '@playwright/test';

test.describe('Test Creation', () => {
  test('organizer can create a new test', async ({ page }) => {
    // Navigate to tests page
    await page.goto('/app/tests');
    
    // Click create test button
    await page.click('button:has-text("Create Test")');
    
    // Fill in test details
    await page.fill('input[name="title"]', 'My New Test');
    await page.fill('textarea[name="description"]', 'Test description');
    
    // Submit form
    await page.click('button[type="submit"]');
    
    // Verify test was created
    await expect(page.locator('text=My New Test')).toBeVisible();
  });
});

Example Participant Test:

import { test, expect } from '@playwright/test';

test.describe('Taking a Test', () => {
  test('participant can submit answers', async ({ page }) => {
    // Navigate to test (using test ID)
    await page.goto('/s/test-id-here');
    
    // Start the test
    await page.click('button:has-text("Start Test")');
    
    // Answer a multiple choice question
    await page.click('text=Option A');
    
    // Submit the test
    await page.click('button:has-text("Submit Test")');
    
    // Verify submission
    await expect(page.locator('text=Test Submitted')).toBeVisible();
  });
});

Test Authentication

E2E tests use a setup project to handle authentication:

Setup Project

The setup project runs first and creates authenticated sessions in e2e/.auth/user.json.

Storage State

Other test projects use the storage state to start with an authenticated session:

{
  name: 'chromium',
  use: {
    storageState: 'e2e/.auth/user.json',
  },
  dependencies: ['setup'],
}

Test Helpers and Fixtures

Reusable test utilities are available in e2e/fixtures/test-helpers.ts to avoid code duplication:

// Example usage in tests
import { createTestWithQuestions } from '../fixtures/test-helpers';

test('grading workflow', async ({ page }) => {
  // Use helper to set up test data
  const testId = await createTestWithQuestions(page, {
    title: 'Grading Test',
    questionCount: 5,
  });
  
  // Continue with test...
});

Best Practices

General Testing Principles

Test behavior, not implementation: Focus on what users see and do
Keep tests independent: Each test should work in isolation
Use descriptive names: Test names should clearly explain what they verify
Avoid hard-coded waits: Use Playwright’s auto-waiting features
Clean up test data: Remove test data after tests complete

Unit Test Best Practices

Test one thing per test case
Mock external dependencies (API calls, database queries)
Use React Testing Library queries in order of preference:
1. getByRole (most accessible)
2. getByLabelText
3. getByPlaceholderText
4. getByText
5. getByTestId (last resort)

E2E Test Best Practices

Test critical user paths first
Use meaningful test data (not “test123”)
Group related tests with test.describe()
Take screenshots on failure (configured automatically)
Run tests locally before pushing

E2E tests automatically capture screenshots, videos, and traces on failure to help debug issues.

Continuous Integration

Tests run automatically in CI environments:

More retries in CI (retries: process.env.CI ? 2 : 1)
No test parallelization due to auth constraints
Development server started automatically
Test reports generated and stored

Troubleshooting

Common Issues

Tests timing out:

Increase timeout in config if testing slow operations
Check if development server is running properly
Verify network requests aren’t failing

Authentication failures:

Ensure setup project ran successfully
Check storage state file exists at e2e/.auth/user.json
Verify Convex environment variables are set

Flaky tests:

Use await expect(locator).toBeVisible() instead of hard-coded waits
Check for race conditions in test logic
Ensure tests clean up after themselves

Debugging Tests

# See the browser while tests run
bun run test:e2e:headed

Future Testing Improvements

Add comprehensive unit test coverage
Implement visual regression testing
Add performance testing for critical paths
Expand E2E test coverage for all features
Add load testing for concurrent users

For questions about testing, visit the GitHub Discussions or open an issue.

Self-Hosting

Technical

Contributing

Testing Stack

Unit Testing with Vitest

Running Unit Tests

Writing Unit Tests

End-to-End Testing with Playwright

Running E2E Tests

E2E Test Configuration

E2E Test Structure

Writing E2E Tests

Test Authentication

Test Helpers and Fixtures

Best Practices

General Testing Principles

Unit Test Best Practices

E2E Test Best Practices

Continuous Integration

Troubleshooting

Common Issues

Debugging Tests

Future Testing Improvements

Build docs developers (and LLMs) love

Self-Hosting

Technical

Contributing

​Testing Stack

​Unit Testing with Vitest

​Running Unit Tests

​Writing Unit Tests

​End-to-End Testing with Playwright

​Running E2E Tests

​E2E Test Configuration

​E2E Test Structure

​Writing E2E Tests

​Test Authentication

​Test Helpers and Fixtures

​Best Practices

​General Testing Principles

​Unit Test Best Practices

​E2E Test Best Practices

​Continuous Integration

​Troubleshooting

​Common Issues

​Debugging Tests

​Future Testing Improvements

Build docs developers (and LLMs) love

Testing Stack

Unit Testing with Vitest

Running Unit Tests

Writing Unit Tests

End-to-End Testing with Playwright

Running E2E Tests

E2E Test Configuration

E2E Test Structure

Writing E2E Tests

Test Authentication

Test Helpers and Fixtures

Best Practices

General Testing Principles

Unit Test Best Practices

E2E Test Best Practices

Continuous Integration

Troubleshooting

Common Issues

Debugging Tests

Future Testing Improvements