Model Providers - SkyDiscover

Overview

SkyDiscover supports any OpenAI-compatible API endpoint. Provider-specific configurations are handled automatically based on model name prefixes.

Supported Providers

OpenAI

GPT-4, GPT-5, o1, o3, o4 models

Google Gemini

Gemini 2.0, 3.0 models

Anthropic

Claude 3.5, 3.7, 4.0 models

DeepSeek

DeepSeek-V3, DeepSeek-Coder

Mistral

Mistral Large, Codestral

Local Models

Ollama, vLLM, any OpenAI-compatible

OpenAI

Setup

export OPENAI_API_KEY="sk-..."

Configuration

llm:
  models:
    - name: "gpt-5"
      weight: 1.0
    - name: "o1"
      reasoning_effort: "high"
      weight: 0.3
  api_base: "https://api.openai.com/v1"
  temperature: 0.7
  max_tokens: 32000

Supported Models

gpt-5 (latest GPT-5)
gpt-4o, gpt-4o-mini
o1, o1-mini, o1-pro
o3, o3-mini
o4, o4-mini

Reasoning Models (o1/o3/o4)

For o-series models, set reasoning effort:

llm:
  models:
    - name: "o1"
      reasoning_effort: "high"  # "low", "medium", "high"
      weight: 1.0
  temperature: 1.0  # o-series uses fixed temperature

Google Gemini

Setup

export GEMINI_API_KEY="AIza..."
# or
export GOOGLE_API_KEY="AIza..."

Configuration

llm:
  models:
    - name: "gemini/gemini-3-pro"
      weight: 1.0
  api_base: "https://generativelanguage.googleapis.com/v1beta/openai/"
  temperature: 0.7
  max_tokens: 32000

Supported Models

gemini/gemini-3-pro (Gemini 3.0 Pro)
gemini/gemini-3-flash (Gemini 3.0 Flash)
gemini/gemini-2.0-flash-exp

The gemini/ prefix is required for auto-detection. SkyDiscover strips it before sending to the API.

Anthropic Claude

Setup

export ANTHROPIC_API_KEY="sk-ant-..."

Configuration

llm:
  models:
    - name: "claude-3-7-sonnet"
      weight: 1.0
  api_base: "https://api.anthropic.com/v1/"
  temperature: 0.7
  max_tokens: 32000

Supported Models

claude-3-7-sonnet
claude-3-5-sonnet
claude-4-opus (when available)

DeepSeek

Setup

export DEEPSEEK_API_KEY="sk-..."

Configuration

llm:
  models:
    - name: "deepseek-chat"
      weight: 1.0
  api_base: "https://api.deepseek.com/v1"
  temperature: 0.7
  max_tokens: 32000

Supported Models

deepseek-chat (DeepSeek-V3)
deepseek-coder

Mistral

Setup

export MISTRAL_API_KEY="..."

Configuration

llm:
  models:
    - name: "mistral-large"
      weight: 1.0
  api_base: "https://api.mistral.ai/v1"

Azure OpenAI

Setup

export AZURE_API_KEY="..."
export AZURE_API_BASE="https://your-resource.openai.azure.com"

Configuration

llm:
  models:
    - name: "gpt-5"  # Your deployment name
      api_base: "https://your-resource.openai.azure.com"
      api_key: ${AZURE_API_KEY}
      weight: 1.0

Local Models

Ollama

Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

Pull a Model

ollama pull qwen2.5-coder:32b

Configure SkyDiscover

llm:
  models:
    - name: "qwen2.5-coder:32b"
      weight: 1.0
  api_base: "http://localhost:11434/v1"
  api_key: "ollama"  # Dummy key

Or via CLI:

skydiscover-run init.py eval.py \
  --api-base http://localhost:11434/v1 \
  -m qwen2.5-coder:32b

vLLM

Install vLLM

pip install vllm

Start vLLM Server

vllm serve deepseek-ai/DeepSeek-Coder-V2-Instruct \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor-parallel-size 4

Configure SkyDiscover

llm:
  models:
    - name: "deepseek-ai/DeepSeek-Coder-V2-Instruct"
      weight: 1.0
  api_base: "http://localhost:8000/v1"
  api_key: "EMPTY"  # vLLM doesn't require auth by default

LM Studio

Install LM Studio

Download from lmstudio.ai

Load a Model

Search and download Qwen/Qwen2.5-Coder-32B-Instruct-GGUF

Start Local Server

In LM Studio: Local Server → Start Server (default port 1234)

Configure SkyDiscover

skydiscover-run init.py eval.py \
  --api-base http://localhost:1234/v1 \
  -m qwen2.5-coder-32b

Multiple Providers

Mix and match providers in a single run:

llm:
  models:
    - name: "gpt-5"
      weight: 0.4
      api_base: "https://api.openai.com/v1"
      api_key: ${OPENAI_API_KEY}
    
    - name: "gemini/gemini-3-pro"
      weight: 0.3
      api_base: "https://generativelanguage.googleapis.com/v1beta/openai/"
      api_key: ${GEMINI_API_KEY}
    
    - name: "claude-3-7-sonnet"
      weight: 0.2
      api_base: "https://api.anthropic.com/v1/"
      api_key: ${ANTHROPIC_API_KEY}
    
    - name: "qwen2.5-coder:32b"
      weight: 0.1
      api_base: "http://localhost:11434/v1"
      api_key: "ollama"

SkyDiscover samples models proportionally to their weights.

Model-Specific Settings

Per-Model Parameters

llm:
  models:
    - name: "gpt-5"
      weight: 0.5
      temperature: 0.8      # Higher creativity
      max_tokens: 40000
    
    - name: "o1"
      weight: 0.5
      temperature: 1.0      # Fixed for o-series
      reasoning_effort: "high"
      max_tokens: 32000
  
  # Defaults for unspecified parameters
  temperature: 0.7
  max_tokens: 32000
  timeout: 600

Custom Client Initialization

For advanced use cases, pass a custom LLM client:

from skydiscover import run_discovery
from skydiscover.config import Config, LLMModelConfig
from openai import AsyncOpenAI

def init_custom_client():
    return AsyncOpenAI(
        base_url="http://localhost:8000/v1",
        api_key="custom-key",
        timeout=600,
    )

config = Config()
config.llm.models = [
    LLMModelConfig(
        name="custom-model",
        init_client=init_custom_client,
        weight=1.0,
    )
]

result = run_discovery(
    evaluator="eval.py",
    config=config,
    iterations=50,
)

Provider Auto-Detection

SkyDiscover automatically detects providers from model names:

Model Name Pattern	Provider	API Base
`gpt-`, `o1`, `o3`, `o4`	OpenAI	`https://api.openai.com/v1`
`gemini-`, `gemini/`	Gemini	`https://generativelanguage.googleapis.com/v1beta/openai/`
`claude-*`	Anthropic	`https://api.anthropic.com/v1/`
`deepseek-*`	DeepSeek	`https://api.deepseek.com/v1`
`mistral-*`	Mistral	`https://api.mistral.ai/v1`
`command-*`	Cohere	`https://api.cohere.com/v1`
Other	OpenAI	`https://api.openai.com/v1`

Override Detection

Use provider/model syntax or explicit api_base:

llm:
  models:
    # Explicit provider prefix
    - name: "gemini/gemini-3-pro"
      weight: 1.0
    
    # Explicit api_base override
    - name: "my-custom-model"
      api_base: "http://localhost:8000/v1"
      weight: 1.0

API Key Priority

SkyDiscover resolves API keys in this order:

Per-model config:

llm:
  models:
    - name: "gpt-5"
      api_key: "sk-..."

Provider environment variable:

export GEMINI_API_KEY="AIza..."
export ANTHROPIC_API_KEY="sk-ant-..."
export DEEPSEEK_API_KEY="sk-..."

Generic OPENAI_API_KEY:
```
export OPENAI_API_KEY="sk-..."
```

Always set provider-specific keys when using multiple providers to avoid auth errors.

Troubleshooting

Authentication Failed

Error: 401 UnauthorizedFix:

Check your API key is correct
Verify the key is exported: echo $OPENAI_API_KEY
Ensure provider-specific key is set (e.g., GEMINI_API_KEY for Gemini)
Try hardcoding the key in config temporarily to isolate the issue

Connection Refused (Local Models)

Error: Connection refused or Failed to connectFix:

Check server is running: curl http://localhost:11434/v1/models
Verify port matches config
For Ollama: ollama serve should show “Listening on 127.0.0.1:11434”
For vLLM: Check logs for startup errors

Model Not Found

Error: The model 'xyz' does not existFix:

For Ollama: ollama list to see available models
For cloud providers: Check spelling and model availability
Verify provider prefix: gemini/gemini-3-pro not just gemini-3-pro

Rate Limit Exceeded

Error: 429 Too Many RequestsFix:

Reduce max_parallel_iterations in config
Add retry delay: retry_delay: 10
Use multiple API keys with separate model entries
Switch to local models for unlimited requests

Best Practices

Use Environment Variables

Never commit API keys. Use env vars or .env files.

# .env
OPENAI_API_KEY=sk-...
GEMINI_API_KEY=AIza...

Mix Local and Cloud

Use cheap local models for exploration, cloud models for exploitation.

models:
  - name: "qwen2.5-coder:32b"
    weight: 0.7  # Exploration
  - name: "gpt-5"
    weight: 0.3  # Exploitation

Set Timeouts

Increase timeout for large models or slow servers.

llm:
  timeout: 1200  # 20 minutes

Monitor Costs

Use gpt-4o-mini or local models during development.

models:
  - name: "gpt-4o-mini"  # $0.15/M tokens
    weight: 1.0

Get Started

Core Concepts

Guides

Examples

Extending

Documentation Index

​Overview

​Supported Providers

OpenAI

Google Gemini

Anthropic

DeepSeek

Mistral

Local Models

​OpenAI

​Setup

​Configuration

​Supported Models

​Reasoning Models (o1/o3/o4)

​Google Gemini

​Setup

​Configuration

​Supported Models

​Anthropic Claude

​Setup

​Configuration

​Supported Models

​DeepSeek

​Setup

​Configuration

​Supported Models

​Mistral

​Setup

​Configuration

​Azure OpenAI

​Setup

​Configuration

​Local Models

​Ollama

​vLLM

​LM Studio

​Multiple Providers

​Model-Specific Settings

​Per-Model Parameters

​Custom Client Initialization

​Provider Auto-Detection

​Override Detection

​API Key Priority

​Troubleshooting

​Best Practices

Use Environment Variables

Mix Local and Cloud

Set Timeouts

Monitor Costs

​Next Steps

Configuration

Running Discovery

Build docs developers (and LLMs) love

Overview

Supported Providers

OpenAI

Setup

Configuration

Supported Models

Reasoning Models (o1/o3/o4)

Google Gemini

Setup

Configuration

Supported Models

Anthropic Claude

Setup

Configuration

Supported Models

DeepSeek

Setup

Configuration

Supported Models

Mistral

Setup

Configuration

Azure OpenAI

Setup

Configuration

Local Models

Ollama

vLLM

LM Studio

Multiple Providers

Model-Specific Settings

Per-Model Parameters

Custom Client Initialization

Provider Auto-Detection

Override Detection

API Key Priority

Troubleshooting

Best Practices

Next Steps