Documentation Index Fetch the complete documentation index at: https://mintlify.com/skydiscover-ai/skydiscover/llms.txt
Use this file to discover all available pages before exploring further.
Overview
SkyDiscover includes a live monitoring dashboard that visualizes solutions as they’re discovered. Watch the evolutionary process unfold in real-time with:
Scatter plot of all programs by score and iteration
Lineage arrows showing parent-child relationships
Code inspection for any program
Metrics charts tracking progress over time
AI-generated summaries of top solutions
Human-in-the-loop feedback interface
Quick Start
Enable Monitoring
Add to your config:
monitor :
enabled : true
host : "127.0.0.1"
port : 8765
Run your discovery:
skydiscover-run init.py eval.py -c config.yaml -s adaevolve -i 100
Open your browser:
The dashboard URL is printed when the run starts: Live monitor: http://localhost:8765/
Configuration
Basic Settings
monitor :
enabled : true # Enable/disable dashboard
host : "127.0.0.1" # Server host (use 0.0.0.0 for remote access)
port : 8765 # Server port
max_solution_length : 10000 # Truncate long solutions in UI
AI Summaries
Generate natural-language summaries of top solutions:
monitor :
enabled : true
# AI summary settings
summary_model : "gpt-5-mini"
summary_api_key : null # Defaults to OPENAI_API_KEY
summary_api_base : "https://api.openai.com/v1"
summary_top_k : 3 # Summarize top-3 programs
summary_interval : 0 # Auto-generate every N programs (0 = manual only)
Usage:
Set summary_interval: 10 to auto-generate summaries every 10 programs
Set summary_interval: 0 for manual generation via dashboard button
Human Feedback
Provide guidance during the run:
human_feedback_enabled : true
human_feedback_file : "human_feedback.md"
human_feedback_mode : "append" # "append" or "replace"
monitor :
enabled : true
Write feedback in Markdown:
# Iteration 15
The circle packing is too dense in the center. Try spreading circles more evenly
across the square.
# Iteration 22
Good improvement! Now focus on optimizing corner utilization.
The LLM sees this feedback in subsequent generations.
Dashboard Features
Scatter Plot
X-axis: Iteration number
Y-axis: Combined score
Color: Score gradient (red = low, green = high)
Arrows: Lineage (parent → child)
Hover: Show program details
Click: Inspect full code
Metrics Panel
Real-time stats:
Best Score: Current best combined_score
Programs Evaluated: Total programs tested
Success Rate: % of valid programs
Average Improvement: Mean score increase per iteration
Runtime: Elapsed time
Code Inspector
Click any point to view:
Full source code with syntax highlighting
All metrics from evaluator
Generation timestamp
Parent program ID
LLM model used
Progress Chart
Tracks score evolution:
Best Score Line: Monotonically increasing
Current Score Points: All evaluated programs
Moving Average: Smoothed trend
AI Summary Panel
Click “Generate Summary” to analyze top programs:
Top 3 Solutions Summary:
1. Program #87 (score: 0.945)
- Uses hexagonal packing in the center
- Optimizes corner utilization with smaller circles
- Achieves 94.5% of theoretical maximum
2. Program #72 (score: 0.923)
- Similar to #87 but with uniform circle sizes
- Simpler implementation, slightly lower score
- Good baseline for further optimization
3. Program #64 (score: 0.901)
- Grid-based approach
- Faster evaluation but lower density
- Useful for understanding trade-offs
Remote Access
SSH Tunnel
If running on a remote server, forward the port:
# On your local machine
ssh -L 8765:localhost:8765 user@remote-server
Then open http://localhost:8765 locally.
Public Access
Only expose the dashboard on trusted networks. It has no authentication.
monitor :
enabled : true
host : "0.0.0.0" # Listen on all interfaces
port : 8765
Access via server IP:
http://192.168.1.100:8765
Programmatic Access
Python API
The monitor can be used programmatically:
from skydiscover import run_discovery
from skydiscover.config import Config, MonitorConfig
config = Config()
config.monitor = MonitorConfig(
enabled = True ,
port = 8765 ,
summary_model = "gpt-4o-mini" ,
summary_interval = 10 , # Auto-generate every 10 programs
)
result = run_discovery(
evaluator = "eval.py" ,
initial_program = "init.py" ,
config = config,
iterations = 100 ,
)
Custom Callbacks
Extend monitoring with custom callbacks:
from skydiscover.extras.monitor import create_monitor_callback, MonitorServer
def my_callback ( event : dict ):
"""Custom event handler."""
if event[ "type" ] == "program_evaluated" :
print ( f "Program { event[ 'id' ] } : score= { event[ 'score' ] } " )
# Send to external logging service, etc.
monitor_server = MonitorServer( port = 8765 )
monitor_server.start()
monitor_callback = create_monitor_callback(monitor_server, custom_handler = my_callback)
# Pass to runner
runner = Runner(
initial_program_path = "init.py" ,
evaluation_file = "eval.py" ,
config = config,
output_dir = "./outputs" ,
)
runner.monitor_callback = monitor_callback
await runner.run( iterations = 100 )
Event Stream
The dashboard receives WebSocket events:
Sent when a new program is generated {
"type" : "program_created" ,
"id" : 42 ,
"iteration" : 15 ,
"parent_id" : 38 ,
"solution" : "def solve(...)..." ,
"model" : "gpt-5"
}
Sent when evaluation completes {
"type" : "program_evaluated" ,
"id" : 42 ,
"score" : 0.87 ,
"metrics" : {
"combined_score" : 0.87 ,
"accuracy" : 0.92 ,
"speed" : 1.3
}
}
Sent after each iteration {
"type" : "iteration_complete" ,
"iteration" : 15 ,
"best_score" : 0.92 ,
"programs_evaluated" : 158
}
Sent when run finishes {
"type" : "discovery_complete" ,
"best_score" : 0.95 ,
"total_iterations" : 100 ,
"runtime" : 3600
}
Offline Viewer
Replay completed runs without re-running discovery:
skydiscover-viewer outputs/adaevolve/circle_packing_0305_1430/
This starts a read-only dashboard showing the saved run data.
The offline viewer is coming in a future release. For now, use the live monitor during runs.
Long solutions slow down the dashboard: monitor :
max_solution_length : 5000 # Truncate at 5000 chars
AI summaries use API credits: monitor :
summary_interval : 0 # Manual generation only
For very fast evaluations, batch events: # Coming in future release
monitor:
event_batch_size: 10
Troubleshooting
Check server logs: INFO: Live monitor: http://localhost:8765/
Verify port is free: Try different port:
WebSocket disconnected — refresh the page. The dashboard automatically reconnects.
Check API key: Verify model: monitor :
summary_model : "gpt-4o-mini" # Must be valid model
Check logs for errors: WARNING: Failed to generate summary: ...
Examples
Basic Monitoring
With AI Summaries
With Human Feedback
Remote Access
monitor :
enabled : true
port : 8765
Next Steps
Configuration Full monitoring configuration reference
Running Discovery Start a monitored discovery run