Configuration - ML Defender

Overview

ML Defender uses JSON configuration files as the single source of truth for all components. All settings are externalized—no hardcoded values.

Via Appia Quality: Configuration files are the law. All runtime behavior is controlled via JSON, enabling deployment without recompilation.

Configuration File Locations

Component	Config File	Description
Firewall Agent	`firewall-acl-agent/config/firewall.json`	IPSet, IPTables, decryption settings
ML Detector	`ml-detector/config/ml_detector_config.json`	Model paths, thresholds, inference settings
Sniffer	`sniffer/config/sniffer.json`	Interface, eBPF settings, feature extraction
etcd Server	`etcd-server/config/etcd-server.json`	Port, heartbeat, auto-restart
RAG Ingester	`rag-ingester/config/rag-ingester.json`	FAISS index, embeddings, log parsing

Firewall ACL Agent Configuration

File: firewall-acl-agent/config/firewall.json

Component Metadata

string

default:"firewall-acl-agent"

Component identifier

string

default:"1.2.1"

Configuration schema version

string

default:"packet_filtering"

Operational mode: packet_filtering or logging_only

Transport Layer

boolean

default:"true"

Enable LZ4 decompression of incoming ML detector events

string

default:"lz4"

Compression algorithm (only lz4 supported)

boolean

default:"true"

Enable ChaCha20-Poly1305 decryption

string

default:"chacha20-poly1305"

Encryption algorithm (AEAD cipher)

boolean

default:"true"

Require crypto token from etcd (fail if unavailable)

etcd Configuration

boolean

default:"true"

Enable etcd client for crypto key exchange

array

default:"[\"localhost:2379\"]"

List of etcd server endpoints

string

default:"/crypto/firewall/tokens"

etcd path where decryption keys are stored

integer

default:"5000"

Connection timeout in milliseconds

integer

default:"30"

Heartbeat interval for service health monitoring

ZeroMQ Configuration

string

default:"tcp://localhost:5572"

ZMQ endpoint to receive encrypted events from ml-detector

string

default:""

ZMQ PUB/SUB topic filter (empty = subscribe to all)

integer

default:"1000"

Receive timeout in milliseconds

integer

default:"1000"

Maximum messages queued before dropping

IPSet Configuration

Critical: IPSet capacity planning is essential. Once full, new IPs are queued but not blocked.

string

default:"ml_defender_blacklist_test"

IPSet name for blocked IPs

string

default:"hash:ip"

IPSet type: hash:ip, hash:net, hash:ip,port

integer

default:"1000"

Maximum IPs in IPSet. Production: Set to 5000-10000

integer

default:"3600"

IP entry timeout in seconds (0 = permanent)

integer

default:"1024"

Hash table size (power of 2 recommended)

boolean

default:"true"

Auto-create IPSet on startup if not exists

boolean

default:"false"

Clear IPSet on startup (use true for testing)

Example: Increase IPSet Capacity

"ipsets": {
  "blacklist": {
    "set_name": "ml_defender_blacklist_prod",
    "max_elements": 10000,
    "timeout": 7200,
    "hash_size": 4096
  }
}

IPTables Configuration

string

default:"ML_DEFENDER_TEST"

IPTables chain name for ML Defender rules

string

default:"ACCEPT"

Default policy for chain: ACCEPT or DROP

boolean

default:"true"

Log blocked packets to syslog

string

default:"ML_DEFENDER_TEST_DROP: "

Prefix for syslog entries

integer

default:"1"

Position in chain (1 = first rule, highest priority)

Batch Processor

integer

default:"10"

Batch size before committing to IPSet

integer

default:"1000"

Max time to wait before flushing batch (ms)

float

default:"0.5"

Minimum ML confidence score to block (0.0-1.0)

Logging

string

default:"debug"

Log level: debug, info, warn, error

string

default:"/vagrant/logs/lab/firewall-agent.log"

Log file path (single source of truth)

integer

default:"10"

Max log file size before rotation

integer

default:"5"

Number of rotated log files to keep

Metrics

boolean

default:"true"

Enable JSON metrics export

integer

default:"30"

Metrics export interval in seconds

string

default:"/vagrant/logs/lab/firewall-metrics.json"

Metrics output file path

Metrics Schema

{
  "timestamp": "2026-02-08T15:30:00Z",
  "zmq_messages_received": 36000,
  "crypto_errors": 0,
  "decompression_errors": 0,
  "protobuf_parse_errors": 0,
  "ipset_successes": 118,
  "ipset_failures": 16681,
  "max_queue_depth": 16690,
  "avg_processing_time_us": 450,
  "cpu_percent": 54.2,
  "memory_rss_mb": 127
}

ML Detector Configuration

File: ml-detector/config/ml_detector_config.json

Model Configuration

string

default:"models/production"

Base directory for ML models

Level 1: Attack Detector

boolean

default:"true"

Enable Level 1 general attack detection

string

default:"level1/level1_attack_detector.onnx"

ONNX model file path (relative to models_base_dir)

integer

default:"23"

Number of input features

float

default:"0.65"

Detection threshold (0.0-1.0)

Level 2: DDoS Detection (Embedded C++20)

boolean

default:"true"

Enable embedded RandomForest DDoS detector

integer

default:"10"

Number of input features

string

default:"RandomForest-Embedded"

Model type (embedded C++20, no ONNX required)

float

default:"0.7"

DDoS detection threshold

Features:

syn_ack_ratio: SYN/ACK packet ratio
packet_symmetry: Bidirectional traffic balance
source_ip_dispersion: Unique source IPs
protocol_anomaly_score: Protocol distribution anomaly
packet_size_entropy: Shannon entropy of packet sizes
traffic_amplification_factor: Amplification attack indicator
flow_completion_rate: TCP handshake completion rate
geographical_concentration: IP geolocation clustering
traffic_escalation_rate: Rate of traffic increase
resource_saturation_score: System resource pressure

Level 2: Ransomware Detection (Embedded C++20)

boolean

default:"true"

Enable embedded RandomForest ransomware detector

integer

default:"10"

Number of input features

float

default:"0.75"

Ransomware detection threshold

Features:

io_intensity: File I/O rate
entropy: File entropy (encrypted data indicator)
resource_usage: CPU/memory consumption
network_activity: C2 communication patterns
file_operations: File modification frequency
process_anomaly: Unusual process behavior
temporal_pattern: Time-series anomalies
access_frequency: File access patterns
data_volume: Data transfer volume
behavior_consistency: Deviation from baseline

Level 3: Traffic Classification (Embedded C++20)

boolean

default:"true"

Enable traffic classification (Internet vs Internal)

integer

default:"10"

Number of input features

float

default:"0.6"

Traffic classification threshold

Level 3: Internal Traffic Analysis (Embedded C++20)

boolean

default:"true"

Enable internal traffic anomaly detection

integer

default:"10"

Number of input features

float

default:"0.65"

Internal traffic threshold

Use Cases: Lateral movement, data exfiltration, service discovery

Transport Configuration

boolean

default:"true"

Enable LZ4 compression of outgoing events

integer

default:"1"

LZ4 compression level (1-12, 1=fastest)

boolean

default:"true"

Enable ChaCha20-Poly1305 encryption

integer

default:"24"

Automatic key rotation interval (hours)

ZeroMQ Output

string

default:"tcp://0.0.0.0:5572"

ZMQ publisher endpoint (binds to this address)

string

default:"PUB"

Socket type: PUB (publish-subscribe)

RAG Logger

boolean

default:"true"

Enable RAG-compatible event logging

string

default:"/vagrant/logs/rag"

Base directory for RAG logs

boolean

default:"true"

Log only detection events (threshold exceeded)

float

default:"0.5"

Minimum score to log event

Sniffer Configuration

File: sniffer/config/sniffer.json

Deployment Mode

string

default:"dual"

Deployment mode:

host-only: Single NIC, host protection
gateway-only: Single NIC, transit traffic inspection
dual: Dual NIC, simultaneous host + gateway
validation: PCAP replay testing

Network Interfaces

string

default:"eth1"

WAN-facing interface for host-based IDS

string

default:"native"

XDP mode: native, offloaded, skb (generic)

string

default:"eth2"

LAN-facing interface for gateway mode

boolean

default:"true"

Enable promiscuous mode (required for gateway)

eBPF Configuration

string

default:"sniffer.bpf.o"

Compiled eBPF object file

string

default:"skb"

XDP attach mode: native, offloaded, skb

integer

default:"1048576"

eBPF ring buffer size (bytes)

integer

default:"500000"

Maximum concurrent flows tracked in kernel

integer

default:"300"

Flow entry timeout (5 minutes)

Fast Detector (Heuristics)

Day 12 Enhancement: Fast path detection runs at <1μs for obvious threats before ML inference.

boolean

default:"true"

Enable fast heuristic detection

float

default:"0.95"

Score for confirmed ransomware patterns

integer

default:"15"

Trigger if >15 external IPs contacted in 30 seconds

integer

default:"10"

Trigger if >10 unique SMB destinations (lateral movement)

float

default:"2.5"

Trigger if DNS query entropy >2.5 (DGA detection)

float

default:"3.0"

Trigger if upload/download ratio >3.0 (exfiltration)

ML Defender Thresholds

float

default:"0.85"

DDoS detection threshold (conservative for PCAP validation)

float

default:"0.90"

Ransomware detection threshold

float

default:"0.80"

Traffic classification threshold

float

default:"0.85"

Internal traffic anomaly threshold

etcd Server Configuration

File: etcd-server/config/etcd-server.json

integer

default:"2379"

etcd client API port

string

default:"0.0.0.0"

Bind address (0.0.0.0 = all interfaces)

integer

default:"4"

Number of worker threads

Heartbeat Monitoring

boolean

default:"true"

Enable service heartbeat monitoring

integer

default:"30"

Heartbeat check interval

integer

default:"3"

Service considered dead after interval * multiplier seconds

Auto-Restart

boolean

default:"true"

Auto-restart failed services

integer

default:"3"

Maximum restart attempts

integer

default:"5"

Delay between restart attempts

string

Command to restart sniffer

cd /vagrant/sniffer/build && sudo ./sniffer -c ../config/sniffer.json

RAG Ingester Configuration

File: rag-ingester/config/rag-ingester.json

Service Identity

string

default:"rag-ingester-default"

Unique service identifier

array

default:"[\"127.0.0.1:2379\"]"

etcd endpoints for service registration

string

default:"ml-detector-default"

Partner ml-detector service ID for crypto key sync

Input Configuration

string

default:"file_watch"

Input source: file_watch or zmq_stream

string

Directory to watch for new log files

string

default:"*.pb.enc"

File pattern to match (encrypted protobuf)

boolean

default:"true"

Files are encrypted (ChaCha20-Poly1305)

boolean

default:"true"

Files are compressed (LZ4)

Embedders

boolean

default:"true"

Enable Chronos temporal embeddings

string

default:"/vagrant/rag-ingester/models/onnx/chronos.onnx"

ONNX model path

integer

default:"83"

Input feature dimension

integer

default:"512"

Embedding dimension

FAISS Configuration

string

default:"Flat"

FAISS index type: Flat, IVF, HNSW

string

default:"L2"

Distance metric: L2, IP (inner product)

string

default:"/shared/faiss_indexes"

Directory for persistent FAISS indexes

integer

default:"1000"

Save index every N events

Example Configurations

Production Firewall Config

Production firewall.json

{
  "ipsets": {
    "blacklist": {
      "set_name": "ml_defender_blacklist_prod",
      "max_elements": 10000,
      "timeout": 7200,
      "hash_size": 4096
    }
  },
  "batch_processor": {
    "batch_size_threshold": 50,
    "min_confidence": 0.7
  },
  "logging": {
    "level": "info",
    "file": "/var/log/ml-defender/firewall.log"
  },
  "metrics": {
    "export_interval_sec": 60
  }
}

Gateway Mode Sniffer Config

Gateway sniffer.json

{
  "deployment": {
    "mode": "dual",
    "host_interface": {
      "name": "eth0",
      "xdp_mode": "native"
    },
    "gateway_interface": {
      "name": "eth1",
      "xdp_mode": "native",
      "promiscuous": true
    },
    "network_settings": {
      "enable_ip_forwarding": true,
      "enable_nat": true
    }
  },
  "profile": "dual_nic"
}

Conservative ML Thresholds

ml_detector_config.json

{
  "ml": {
    "thresholds": {
      "level1_attack": 0.8,
      "level2_ddos": 0.85,
      "level2_ransomware": 0.9,
      "level3_web": 0.75,
      "level3_internal": 0.85
    }
  }
}

Configuration Best Practices

Single Source of Truth: Never hardcode values. All settings in JSON.

Test Before Production: Validate configs in lab with dry_run: true

Monitor Metrics: Always enable metrics export for capacity planning

IPSet Capacity: Plan for 2x expected peak blocked IPs

Threshold Tuning: Start conservative (0.8+), lower based on false positive rate

Crypto Seed Exchange

ML Defender uses ChaCha20-Poly1305 authenticated encryption for all inter-component communication.

How It Works

ml-detector generates a random 256-bit key on startup
Key is stored in etcd at /crypto/ml-detector/tokens
firewall-acl-agent reads key from etcd at /crypto/firewall/tokens
Both components use the same key for encryption/decryption
Keys rotate every 24 hours (configurable)

Verify Crypto Sync

# Check etcd keys
etcdctl get --prefix /crypto

# Should show:
# /crypto/ml-detector/tokens: <base64-encoded-key>
# /crypto/firewall/tokens: <base64-encoded-key>

Manual Key Rotation

# Restart ml-detector to generate new key
sudo pkill ml-detector
cd ml-detector/build && ./ml-detector -c ../config/ml_detector_config.json

# Restart firewall-agent to sync new key
sudo pkill firewall-acl-agent
cd firewall-acl-agent/build && sudo ./firewall-acl-agent -c ../config/firewall.json

Next Steps

Quick Start

Launch ML Defender with default configs

Deployment

Production deployment strategies

Monitoring

Metrics, alerts, and observability

Troubleshooting

Common issues and solutions

​Overview

​Configuration File Locations

​Firewall ACL Agent Configuration

​Component Metadata

​Transport Layer

​etcd Configuration

​ZeroMQ Configuration

​IPSet Configuration

​Example: Increase IPSet Capacity

​IPTables Configuration

​Batch Processor

​Logging

​Metrics

​Metrics Schema

​ML Detector Configuration

​Model Configuration

​Level 1: Attack Detector

​Level 2: DDoS Detection (Embedded C++20)

​Level 2: Ransomware Detection (Embedded C++20)

​Level 3: Traffic Classification (Embedded C++20)

​Level 3: Internal Traffic Analysis (Embedded C++20)

​Transport Configuration

​ZeroMQ Output

​RAG Logger

​Sniffer Configuration

​Deployment Mode

​Network Interfaces

​eBPF Configuration

​Fast Detector (Heuristics)

​ML Defender Thresholds

​etcd Server Configuration

​Heartbeat Monitoring

​Auto-Restart

​RAG Ingester Configuration

​Service Identity

​Input Configuration

​Embedders

​FAISS Configuration

​Example Configurations

​Production Firewall Config

​Gateway Mode Sniffer Config

​Conservative ML Thresholds

​Configuration Best Practices

​Crypto Seed Exchange

​How It Works

​Verify Crypto Sync

​Manual Key Rotation

​Next Steps

Quick Start

Deployment

Monitoring

Troubleshooting

Overview

Configuration File Locations

Firewall ACL Agent Configuration

Component Metadata

Transport Layer

etcd Configuration

ZeroMQ Configuration

IPSet Configuration

Example: Increase IPSet Capacity

IPTables Configuration

Batch Processor

Logging

Metrics

Metrics Schema

ML Detector Configuration

Model Configuration

Level 1: Attack Detector

Level 2: DDoS Detection (Embedded C++20)

Level 2: Ransomware Detection (Embedded C++20)

Level 3: Traffic Classification (Embedded C++20)

Level 3: Internal Traffic Analysis (Embedded C++20)

Transport Configuration

ZeroMQ Output

RAG Logger

Sniffer Configuration

Deployment Mode

Network Interfaces

eBPF Configuration

Fast Detector (Heuristics)

ML Defender Thresholds

etcd Server Configuration

Heartbeat Monitoring

Auto-Restart

RAG Ingester Configuration

Service Identity

Input Configuration

Embedders

FAISS Configuration

Example Configurations

Production Firewall Config

Gateway Mode Sniffer Config

Conservative ML Thresholds

Configuration Best Practices

Crypto Seed Exchange

How It Works

Verify Crypto Sync

Manual Key Rotation

Next Steps