Troubleshooting - Brainbox

This guide covers common issues you may encounter when self-hosting Brainbox and how to resolve them.

Database Connection Issues

Error: “Connection refused” or “ECONNREFUSED”

Symptoms:

Server fails to start
Error message: Error connecting to PostgreSQL: connect ECONNREFUSED

Causes and Solutions:

Verify PostgreSQL is running

docker compose ps postgres

# Should show "Up" status
# If not running:
docker compose up -d postgres

Check connection string

Verify POSTGRES_URL in your environment configuration:

Docker Compose

# Should use service name, not localhost
POSTGRES_URL=postgres://user:pass@postgres:5432/colanode_db
# NOT: localhost or 127.0.0.1

Kubernetes

# Should use Kubernetes service name
POSTGRES_URL=postgres://user:pass@my-brainbox-postgresql:5432/colanode_db

Verify network connectivity

# Services must be on the same network
docker network inspect brainbox_network

Check password and credentials

Ensure password in POSTGRES_URL matches the database password:

Docker Compose

# Check environment variables
docker compose config | grep POSTGRES

Error: “password authentication failed”

Solution: Password mismatch between server config and database.

# Stop services
docker compose down

# Edit .env to match passwords
nano .env

# Ensure POSTGRES_PASSWORD matches in:
# 1. postgres service environment
# 2. server POSTGRES_URL

# Restart
docker compose up -d

Error: “database does not exist”

Solution: Database not created or wrong database name.

# Connect to PostgreSQL
docker exec -it brainbox_postgres psql -U postgres

# List databases
\l

# Create database if missing
CREATE DATABASE colanode_db;

# Grant permissions
GRANT ALL PRIVILEGES ON DATABASE colanode_db TO colanode_user;

# Exit
\q

Database migration failures

Error: “Migration failed” or “relation already exists” Solution:

Check migration status

docker compose logs server | grep migration

Reset database (development only)

This will delete all data. Only use in development.

Docker Compose

# Stop server
docker compose stop server

# Connect to database
docker exec -it brainbox_postgres psql -U colanode_user -d colanode_db

# Drop all tables
DROP SCHEMA public CASCADE;
CREATE SCHEMA public;

# Restart server (will run migrations)
docker compose start server

Manual migration fix

Check server logs for specific migration errors:

docker compose logs server | grep -A 10 "Migration error"

Redis Connection Issues

Error: “Redis connection refused”

Solution:

Verify Redis is running

docker compose ps valkey
docker compose logs valkey

Check Redis URL format

# Correct format with password
REDIS_URL=redis://:your_password@valkey:6379/0

# Note the colon before password: ":password"
# NOT: redis://valkey:6379/0 (missing password)

Test Redis connection

docker exec -it brainbox_valkey valkey-cli -a your_password ping
# Should return: PONG

Error: “NOAUTH Authentication required”

Solution: Redis password not provided or incorrect.

# Docker Compose: Check .env file
VALKEY_PASSWORD=your_valkey_password

# Ensure REDIS_URL includes password
REDIS_URL=redis://:your_valkey_password@valkey:6379/0

Storage (S3/MinIO) Issues

Error: “Access Denied” or “Invalid credentials”

Solution:

Verify MinIO credentials

# Check .env file
cat .env | grep MINIO

# Credentials must match between:
# 1. MinIO service (MINIO_ROOT_USER/PASSWORD)
# 2. Server config (STORAGE_S3_ACCESS_KEY/SECRET_KEY)

Test MinIO access

Access MinIO console:

Docker Compose: http://localhost:9001
Kubernetes: kubectl port-forward svc/my-brainbox-minio 9001:9001

Verify bucket exists

MinIO console → Buckets → should see “brainbox” bucketIf missing:

Docker Compose

docker compose restart minio
# The entrypoint command creates the bucket on startup

Error: “Bucket does not exist”

Solution: Create the bucket manually:

Access MinIO console (http://localhost:9001)
Login with MINIO_ROOT_USER/PASSWORD
Click "Create Bucket"
Name: "brainbox"
Click "Create"

Error: “Connection timeout” to S3 endpoint

Solution: Check endpoint URL format:

# MinIO (Docker/Kubernetes): Use http://, not https://
STORAGE_S3_ENDPOINT=http://minio:9000
STORAGE_S3_FORCE_PATH_STYLE=true

# AWS S3: Use https://
STORAGE_S3_ENDPOINT=https://s3.amazonaws.com
STORAGE_S3_FORCE_PATH_STYLE=false

Port Conflicts

Error: “Port already in use” or “Address already in use”

Solution:

Find process using the port

# Check port 4000 (web)
lsof -i :4000

# Check port 3000 (server)
lsof -i :3000

# Check port 5432 (postgres)
lsof -i :5432

Stop conflicting process or change port

Option 1: Stop the conflicting process

# Kill process by PID (from lsof output)
kill -9 <PID>

Option 2: Change Brainbox ports in docker-compose.yaml:

docker-compose.yaml

services:
  web:
    ports:
      - '8080:80'  # Change 4000 to 8080
  
  server:
    ports:
      - '8000:3000'  # Change 3000 to 8000

Container/Pod Issues

Container keeps restarting

Diagnosis:

# Check container status
docker compose ps

# View logs
docker compose logs server --tail=100

# Check exit code
docker inspect brainbox_server | grep -A 5 State

Common causes:

Environment variable errors: Check logs for configuration errors
Dependency not ready: Database or Redis not available
Out of memory: Check resource limits
Failed health checks: Server not responding on expected port

Out of memory errors

Solution:

# Increase memory limits in docker-compose.yaml
services:
  server:
    deploy:
      resources:
        limits:
          memory: 2G
        reservations:
          memory: 1G

WebSocket Connection Issues

Error: “WebSocket connection failed”

Symptoms:

Real-time updates not working
“Connection lost” messages in UI
Browser console shows WebSocket errors

Solution:

Check reverse proxy WebSocket support

If using nginx:

nginx.conf

location /socket.io/ {
    proxy_pass http://localhost:3000;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header Host $host;
}

Verify server mode for multiple replicas

If running multiple server instances:

SERVER_MODE=cluster  # Required for WebSocket sync across instances

Check firewall rules

Ensure WebSocket ports are not blocked by firewall.

Performance Issues

Slow queries or timeouts

Solution:

Check database performance

Docker Compose

docker exec -it brainbox_postgres psql -U colanode_user -d colanode_db

-- Check slow queries
SELECT pid, now() - pg_stat_activity.query_start AS duration, query 
FROM pg_stat_activity 
WHERE state = 'active' 
ORDER BY duration DESC;

-- Check database size
SELECT pg_size_pretty(pg_database_size('colanode_db'));

Increase resource limits

services:
  postgres:
    deploy:
      resources:
        limits:
          memory: 4G
          cpus: '2'

Enable query logging

LOGGING_LEVEL=debug

Check logs for slow database queries.

High memory usage

Solution:

Monitor container memory:
```
docker stats
```
Reduce concurrent connections: Configure PostgreSQL connection pool limits

Enable Redis eviction:

docker exec -it brainbox_valkey valkey-cli
CONFIG SET maxmemory 2gb
CONFIG SET maxmemory-policy allkeys-lru

Update/Upgrade Issues

Migration errors after update

Solution:

Check migration logs

docker compose logs server | grep -i migration

Rollback if needed

# Stop services
docker compose down

# Restore database backup
gunzip -c backup.sql.gz | docker exec -i brainbox_postgres psql -U colanode_user -d colanode_db

# Use previous image version
# Edit docker-compose.yaml to specify old tag
image: ghcr.io/your-username/brainbox-server:v1.0.0

docker compose up -d

“Image pull” errors

Error: Failed to pull image from registry Solution:

Verify image exists

# Check if image tag exists
docker pull ghcr.io/your-username/brainbox-server:latest

Check registry authentication

# Login to GHCR
echo $GITHUB_TOKEN | docker login ghcr.io -u username --password-stdin

Update image tag

Use specific version instead of latest:

image: ghcr.io/your-username/brainbox-server:v1.0.0

Getting Help

If you’re still experiencing issues:

Collect diagnostic information

# System info
docker version
docker compose version

# Service status
docker compose ps

# Logs
docker compose logs > brainbox-logs.txt

# Container inspection
docker inspect brainbox_server > server-inspect.txt

Report the issue

Check existing issues: GitHub Issues
Create new issue with:
- Deployment method (Docker/Kubernetes)
- Brainbox version
- Error messages and logs
- Steps to reproduce
- Environment details (OS, Docker version, etc.)

Community support

GitHub Discussions
Documentation updates and examples

Preventive Maintenance

Regular backups

backup.sh

#!/bin/bash
# Automated backup script

BACKUP_DIR="/backups/brainbox"
DATE=$(date +%Y%m%d_%H%M%S)
mkdir -p $BACKUP_DIR

# PostgreSQL backup
docker exec brainbox_postgres pg_dump -U colanode_user colanode_db | \
  gzip > "$BACKUP_DIR/postgres_$DATE.sql.gz"

# MinIO backup (optional)
docker exec brainbox_minio mc mirror /data/brainbox "$BACKUP_DIR/minio_$DATE/"

# Keep last 30 days
find $BACKUP_DIR -name "*.sql.gz" -mtime +30 -delete

Monitoring

Set up basic monitoring:

healthcheck.sh

#!/bin/bash
# Health check script

if ! curl -f http://localhost:4000 > /dev/null 2>&1; then
  echo "Brainbox web is down!"
  # Send alert (email, Slack, etc.)
fi

if ! docker exec brainbox_postgres pg_isready -U colanode_user > /dev/null 2>&1; then
  echo "PostgreSQL is down!"
fi

Add to crontab:

*/5 * * * * /path/to/healthcheck.sh

Next Steps

Configuration Reference

Review all configuration options

Docker Setup

Docker Compose deployment guide

Kubernetes Setup

Kubernetes deployment guide

Get Started

Core Features

Self-Hosting

Architecture

​Database Connection Issues

​Error: “Connection refused” or “ECONNREFUSED”

​Error: “password authentication failed”

​Error: “database does not exist”

​Database migration failures

​Redis Connection Issues

​Error: “Redis connection refused”

​Error: “NOAUTH Authentication required”

​Storage (S3/MinIO) Issues

​Error: “Access Denied” or “Invalid credentials”

​Error: “Bucket does not exist”

​Error: “Connection timeout” to S3 endpoint

​Port Conflicts

​Error: “Port already in use” or “Address already in use”

​Container/Pod Issues

​Container keeps restarting

​Out of memory errors

​WebSocket Connection Issues

​Error: “WebSocket connection failed”

​Performance Issues

​Slow queries or timeouts

​High memory usage

​Update/Upgrade Issues

​Migration errors after update

​“Image pull” errors

​Getting Help

​Collect diagnostic information

​Report the issue

​Community support

​Preventive Maintenance

​Regular backups

​Monitoring

​Next Steps

Configuration Reference

Docker Setup

Kubernetes Setup

Build docs developers (and LLMs) love

Database Connection Issues

Error: “Connection refused” or “ECONNREFUSED”

Error: “password authentication failed”

Error: “database does not exist”

Database migration failures

Redis Connection Issues

Error: “Redis connection refused”

Error: “NOAUTH Authentication required”

Storage (S3/MinIO) Issues

Error: “Access Denied” or “Invalid credentials”

Error: “Bucket does not exist”

Error: “Connection timeout” to S3 endpoint

Port Conflicts

Error: “Port already in use” or “Address already in use”

Container/Pod Issues

Container keeps restarting

Out of memory errors

WebSocket Connection Issues

Error: “WebSocket connection failed”

Performance Issues

Slow queries or timeouts

High memory usage

Update/Upgrade Issues

Migration errors after update

“Image pull” errors

Getting Help

Collect diagnostic information

Report the issue

Community support

Preventive Maintenance

Regular backups

Monitoring

Next Steps