Deployment Guide

This guide covers deploying the Runtime service to production, including containerization, security considerations, and best practices.

Deployment Options

Docker Container

Deploy as a containerized application

Kubernetes

Orchestrate at scale with K8s

Traditional Server

Run directly on a VM or bare metal

Cloud Platforms

Deploy to AWS, GCP, or Azure

Building for Production

Create Production JAR

Build an optimized JAR file for deployment:

cd backend
mvn clean package -DskipTests

The compiled JAR will be located at:

target/engine-0.0.1-SNAPSHOT.jar

Build Docker Image

Create a Dockerfile in the backend directory:

Dockerfile

FROM eclipse-temurin:21-jre

WORKDIR /app

# Copy the JAR file
COPY target/engine-0.0.1-SNAPSHOT.jar app.jar

# Expose application port
EXPOSE 8081

# Run the application
ENTRYPOINT ["java", "-jar", "app.jar"]

Build the image:

docker build -t runtime-service:latest .

Docker Deployment

Docker Compose Setup

Create a docker-compose.yml file:

docker-compose.yml

version: '3.8'

services:
  runtime-service:
    image: runtime-service:latest
    container_name: runtime-api
    ports:
      - "8081:8081"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - SPRING_PROFILES_ACTIVE=production
      - DOCKER_HOST=unix:///var/run/docker.sock
    restart: unless-stopped
    networks:
      - runtime-network

networks:
  runtime-network:
    driver: bridge

Mounting the Docker socket (/var/run/docker.sock) gives the container access to the Docker daemon. This is required for code execution but should be secured properly.

Run with Docker Compose

docker-compose up -d

View logs:

docker-compose logs -f runtime-service

Kubernetes Deployment

Create Deployment Manifest

k8s-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: runtime-service
  labels:
    app: runtime-service
spec:
  replicas: 3
  selector:
    matchLabels:
      app: runtime-service
  template:
    metadata:
      labels:
        app: runtime-service
    spec:
      containers:
      - name: runtime-api
        image: runtime-service:latest
        ports:
        - containerPort: 8081
        env:
        - name: SPRING_PROFILES_ACTIVE
          value: "production"
        - name: DOCKER_HOST
          value: "tcp://docker-daemon:2375"
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "1Gi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /actuator/health
            port: 8081
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /actuator/health
            port: 8081
          initialDelaySeconds: 20
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: runtime-service
spec:
  selector:
    app: runtime-service
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8081
  type: LoadBalancer

Deploy to Kubernetes:

kubectl apply -f k8s-deployment.yaml

For Kubernetes deployments, you’ll need a Docker-in-Docker (DinD) sidecar or a dedicated Docker daemon service to handle code execution containers.

Environment Configuration

Production Application Properties

Create application-production.properties:

application-production.properties

spring.application.name=runtime-service
server.port=8081

# Docker configuration
docker.host=unix:///var/run/docker.sock

# Logging
logging.level.root=INFO
logging.level.com.runtime=DEBUG
logging.file.name=/var/log/runtime-service.log

# Resource limits (already hardcoded in DockerExecutorUtil.java)
# Memory: 128MB per container
# CPU: 0.5 cores per container

Environment Variables

Key environment variables for deployment:

DOCKER_HOST

string

default:"tcp://localhost:2375"

Docker daemon connection URL

SPRING_PROFILES_ACTIVE

string

default:"default"

Spring Boot active profile (e.g., production)

SERVER_PORT

number

default:8081

Port on which the service listens

JAVA_OPTS

string

JVM options (e.g., -Xmx2g -Xms512m)

Security Considerations

The Runtime service executes arbitrary code. Implement strict security measures in production.

Docker Security

Isolate Docker Daemon

Run the Runtime service with its own dedicated Docker daemon, not the host’s primary daemon:

docker run -d --name docker-daemon \
  --privileged \
  docker:dind

Enable Resource Limits

Resource limits are hardcoded in DockerExecutorUtil.java:

Memory: 128 MB per container
CPU: 0.5 cores per container

These prevent resource exhaustion attacks.

Network Isolation

Execution containers should not have network access:Update DockerExecutorUtil.java to add:

.withNetworkMode("none")

Container Cleanup

The service automatically removes containers after execution (line 86 in DockerExecutorUtil.java):

dockerClient.removeContainerCmd(containerId).exec();

API Security

Add Authentication

Implement Spring Security to protect the API:

pom.xml

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-security</artifactId>
</dependency>

Configure API key or JWT authentication.

Rate Limiting

Implement rate limiting to prevent abuse:

@RateLimit(requests = 10, per = "1m")
public ApiResponse<ExecutionResult> execute(...) {
    // ...
}

Input Validation

Add code size limits and content validation:

if (code.length() > 10_000) {
    throw new IllegalArgumentException("Code exceeds maximum length");
}

Firewall Rules

Only expose necessary ports:

# Allow HTTP traffic to API
sudo ufw allow 8081/tcp

# Block direct Docker daemon access
sudo ufw deny 2375/tcp

Monitoring and Logging

Add Spring Boot Actuator

Enable health checks and metrics:

pom.xml

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

application.properties

management.endpoints.web.exposure.include=health,metrics,info
management.endpoint.health.show-details=always

Monitor Docker Containers

Track active execution containers:

watch -n 1 'docker ps --filter "ancestor=python:3.11" \
  --filter "ancestor=eclipse-temurin:17" \
  --filter "ancestor=gcc:latest" \
  --filter "ancestor=node:18"'

Application Logs

The service logs key events (from DockerExecutorUtil.java):

Docker container started!!!
Docker container removed!!!

Configure structured logging for production:

logging.pattern.console=%d{yyyy-MM-dd HH:mm:ss} - %msg%n
logging.file.name=/var/log/runtime-service.log
logging.file.max-size=10MB
logging.file.max-history=10

Scaling Considerations

Horizontal Scaling

The service is stateless and can be scaled horizontally:

# Docker Compose
docker-compose up -d --scale runtime-service=3

# Kubernetes
kubectl scale deployment runtime-service --replicas=5

Load Balancing

Use NGINX or a cloud load balancer:

nginx.conf

upstream runtime_backend {
    least_conn;
    server runtime-1:8081;
    server runtime-2:8081;
    server runtime-3:8081;
}

server {
    listen 80;
    
    location /api/ {
        proxy_pass http://runtime_backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Resource Planning

Each code execution uses:

128 MB memory (container limit)
0.5 CPU cores (container limit)
Temporary disk space (~1-10 MB)

Plan server resources based on expected concurrent executions.

Health Checks

Implement a health check endpoint:

curl http://localhost:8081/actuator/health

Expected response:

{
  "status": "UP"
}

Backup and Recovery

The Runtime service is stateless. No data backup is required. However, ensure Docker images are available:

# Save images
docker save python:3.11 eclipse-temurin:17 gcc:latest node:18 > runtime-images.tar

# Load images on new server
docker load < runtime-images.tar

Performance Optimization

Pre-pull Images

Pull all language images before deployment:

docker pull python:3.11
docker pull eclipse-temurin:17
docker pull gcc:latest
docker pull node:18

Increase JVM Memory

For high-load environments:

java -Xmx2g -Xms512m -jar app.jar

Use Persistent Temp Directory

Mount a fast SSD volume for temporary files:

volumes:
  - /mnt/fast-ssd/tmp:/tmp

Troubleshooting Production Issues

High memory usage

Check for containers not being cleaned up:

docker ps -a | grep -E "python|java|gcc|node"

Force cleanup:

docker container prune -f

Slow execution times

Verify Docker images are pre-pulled
Check disk I/O performance
Monitor CPU and memory availability
Review execution time logs in ExecutionResult.executionTime

Docker daemon unreachable

Verify Docker daemon is running:

docker info

Check DOCKER_HOST configuration matches daemon address.

Next Steps

Security Architecture

Deep dive into security features

API Reference

Complete API documentation

Supported Languages

Language-specific configurations

Docker Execution

How containers are managed

Get Started

Architecture

Guides

​Deployment Options

Docker Container

Kubernetes

Traditional Server

Cloud Platforms

​Building for Production

​Create Production JAR

​Build Docker Image

​Docker Deployment

​Docker Compose Setup

​Run with Docker Compose

​Kubernetes Deployment

​Create Deployment Manifest

​Environment Configuration

​Production Application Properties

​Environment Variables

​Security Considerations

​Docker Security

​API Security

​Firewall Rules

​Monitoring and Logging

​Add Spring Boot Actuator

​Monitor Docker Containers

​Application Logs

​Scaling Considerations

​Horizontal Scaling

​Load Balancing

​Resource Planning

​Health Checks

​Backup and Recovery

​Performance Optimization

​Troubleshooting Production Issues

​Next Steps

Security Architecture

API Reference

Supported Languages

Docker Execution

Build docs developers (and LLMs) love

Deployment Options

Building for Production

Create Production JAR

Build Docker Image

Docker Deployment

Docker Compose Setup

Run with Docker Compose

Kubernetes Deployment

Create Deployment Manifest

Environment Configuration

Production Application Properties

Environment Variables

Security Considerations

Docker Security

API Security

Firewall Rules

Monitoring and Logging

Add Spring Boot Actuator

Monitor Docker Containers

Application Logs

Scaling Considerations

Horizontal Scaling

Load Balancing

Resource Planning

Health Checks

Backup and Recovery

Performance Optimization

Troubleshooting Production Issues

Next Steps