Model Router Architecture

Overview

The Model Router is responsible for transforming incoming model requests from various API formats (OpenAI, Anthropic, Gemini) into the appropriate upstream format and vice versa.

Key Responsibilities

Model ID Mapping - Transform client model IDs to upstream IDs
Protocol Conversion - Convert request/response between formats
Dynamic Forwarding - Route deprecated models to new ones
Parameter Adaptation - Adjust parameters for upstream compatibility

Model Mapping System

Location: src-tauri/src/proxy/common/model_mapping.rs

Standard ID Normalization

pub fn normalize_to_standard_id(model_name: &str) -> Option<String> {
    // Remove suffixes like -thinking, -high, -low
    let clean_name = model_name
        .trim_end_matches("-thinking")
        .trim_end_matches("-high")
        .trim_end_matches("-low")
        .to_lowercase();
    
    // Map to standard ID
    match clean_name.as_str() {
        "claude-sonnet-4-6" | "claude-sonnet-4.6" => Some("claude-sonnet-4-6"),
        "claude-opus-4-6" | "claude-opus-4.6" => Some("claude-opus-4-6"),
        "gemini-3.1-pro" | "gemini-pro-high" => Some("gemini-3.1-pro"),
        "gemini-3.1-flash" | "gemini-flash-high" => Some("gemini-3.1-flash"),
        "gemini-3-pro-image" => Some("gemini-3-pro-image"),
        _ => None,
    }
}

Custom Mapping

Users can define custom model ID mappings in the configuration:

pub struct AppState {
    pub custom_mapping: Arc<RwLock<HashMap<String, String>>>,
}

pub async fn apply_model_mapping(
    model: String,
    state: &AppState,
) -> String {
    let mapping = state.custom_mapping.read().await;
    
    mapping.get(&model)
        .cloned()
        .unwrap_or(model)
}

Example Configuration:

{
  "custom_mapping": {
    "gpt-4": "gemini-3.1-pro",
    "gpt-4-turbo": "gemini-3.1-pro-high",
    "claude-opus": "claude-opus-4-6"
  }
}

Dynamic Forwarding Rules

Purpose: Automatically redirect deprecated models to new ones Location: src-tauri/src/proxy/token_manager.rs:511

// Load forwarding rules from account quota data
if let Some(rules) = account["quota"]["model_forwarding_rules"].as_object() {
    for (old_model, new_model) in rules {
        update_dynamic_forwarding_rules(
            old_model.to_string(),
            new_model.as_str().unwrap().to_string(),
        );
    }
}

Global Forwarding Table:

static DYNAMIC_FORWARDING_RULES: OnceLock<RwLock<HashMap<String, String>>> = OnceLock::new();

pub fn update_dynamic_forwarding_rules(old_model: String, new_model: String) {
    let rules = DYNAMIC_FORWARDING_RULES.get_or_init(|| RwLock::new(HashMap::new()));
    rules.write().unwrap().insert(old_model, new_model);
}

pub fn apply_forwarding_rules(model: &str) -> String {
    let rules = DYNAMIC_FORWARDING_RULES.get_or_init(|| RwLock::new(HashMap::new()));
    rules.read().unwrap()
        .get(model)
        .cloned()
        .unwrap_or_else(|| model.to_string())
}

Protocol Conversion

Request Flow

Client Request (OpenAI/Claude/Gemini)
  → Model Mapping
    → Custom Mapping
      → Dynamic Forwarding
        → Protocol Conversion
          → Upstream Request (Google/Anthropic API)

OpenAI → Gemini Conversion

Location: src-tauri/src/proxy/handlers/openai.rs

pub fn convert_openai_to_gemini(
    request: OpenAIChatRequest,
    target_model: &str,
) -> Result<GeminiRequest, AppError> {
    // 1. Convert messages
    let contents = request.messages.iter()
        .map(|msg| GeminiContent {
            role: match msg.role.as_str() {
                "system" => "user",  // Gemini doesn't have system role
                "assistant" => "model",
                "user" => "user",
                _ => "user",
            },
            parts: convert_message_content(&msg.content),
        })
        .collect();
    
    // 2. Handle system messages
    let system_instruction = request.messages.iter()
        .find(|m| m.role == "system")
        .map(|m| m.content.clone());
    
    // 3. Convert generation config
    let generation_config = GeminiGenerationConfig {
        temperature: request.temperature,
        max_output_tokens: request.max_tokens,
        top_p: request.top_p,
        top_k: request.top_k,
        stop_sequences: request.stop,
        response_mime_type: request.response_format
            .and_then(|f| if f.type_ == "json_object" {
                Some("application/json")
            } else { None }),
    };
    
    // 4. Convert tools (function calling)
    let tools = request.tools.map(|tools| {
        tools.into_iter().map(convert_openai_tool_to_gemini).collect()
    });
    
    Ok(GeminiRequest {
        contents,
        system_instruction,
        generation_config: Some(generation_config),
        tools,
        safety_settings: None,
    })
}

Gemini → OpenAI Conversion

pub fn convert_gemini_to_openai(
    response: GeminiResponse,
    model: &str,
) -> Result<OpenAIResponse, AppError> {
    let candidate = response.candidates
        .first()
        .ok_or(AppError::InvalidResponse("No candidates"))?;
    
    let content = candidate.content.parts.iter()
        .filter_map(|part| part.text.clone())
        .collect::<Vec<_>>()
        .join("");
    
    // Extract tool calls
    let tool_calls = candidate.content.parts.iter()
        .filter_map(|part| part.function_call.as_ref())
        .enumerate()
        .map(|(i, fc)| OpenAIToolCall {
            id: format!("call_{}", i),
            type_: "function".to_string(),
            function: OpenAIFunction {
                name: fc.name.clone(),
                arguments: serde_json::to_string(&fc.args).unwrap(),
            },
        })
        .collect::<Vec<_>>();
    
    // Calculate token usage
    let usage = OpenAIUsage {
        prompt_tokens: response.usage_metadata.prompt_token_count,
        completion_tokens: response.usage_metadata.candidates_token_count,
        total_tokens: response.usage_metadata.total_token_count,
    };
    
    Ok(OpenAIResponse {
        id: format!("chatcmpl-{}", uuid::Uuid::new_v4()),
        object: "chat.completion".to_string(),
        created: chrono::Utc::now().timestamp(),
        model: model.to_string(),
        choices: vec![OpenAIChoice {
            index: 0,
            message: OpenAIMessage {
                role: "assistant".to_string(),
                content: if content.is_empty() { None } else { Some(content) },
                tool_calls: if tool_calls.is_empty() { None } else { Some(tool_calls) },
            },
            finish_reason: map_finish_reason(&candidate.finish_reason),
        }],
        usage: Some(usage),
    })
}

Claude → Gemini Conversion

Location: src-tauri/src/proxy/handlers/claude.rs

pub fn convert_claude_to_gemini(
    request: ClaudeRequest,
    target_model: &str,
) -> Result<GeminiRequest, AppError> {
    // 1. Extract system message
    let system_instruction = if let Some(system) = request.system {
        Some(system)
    } else {
        None
    };
    
    // 2. Convert messages
    let contents = request.messages.iter()
        .map(|msg| GeminiContent {
            role: if msg.role == "user" { "user" } else { "model" },
            parts: match &msg.content {
                ClaudeContent::Text(text) => vec![GeminiPart::text(text)],
                ClaudeContent::Array(parts) => parts.iter()
                    .map(convert_claude_content_block)
                    .collect(),
            },
        })
        .collect();
    
    // 3. Handle thinking mode
    let thinking_config = if target_model.contains("thinking") {
        Some(GeminiThinkingConfig {
            thinking_budget: request.thinking
                .and_then(|t| t.budget_tokens)
                .unwrap_or(24576),
        })
    } else {
        None
    };
    
    // 4. Generation config
    let generation_config = GeminiGenerationConfig {
        temperature: request.temperature,
        max_output_tokens: request.max_tokens,
        top_p: request.top_p,
        top_k: request.top_k,
        stop_sequences: request.stop_sequences,
        thinking_config,
    };
    
    Ok(GeminiRequest {
        contents,
        system_instruction,
        generation_config: Some(generation_config),
        tools: request.tools.map(convert_claude_tools),
        safety_settings: None,
    })
}

Streaming Response Handling

SSE Stream Conversion

OpenAI Streaming:

pub async fn stream_gemini_to_openai(
    gemini_stream: impl Stream<Item = Result<GeminiChunk, Error>>,
    model: String,
) -> impl Stream<Item = Result<Bytes, Error>> {
    let stream_id = format!("chatcmpl-{}", uuid::Uuid::new_v4());
    
    gemini_stream.map(move |chunk| {
        let chunk = chunk?;
        
        // Convert chunk to OpenAI format
        let openai_chunk = OpenAIStreamChunk {
            id: stream_id.clone(),
            object: "chat.completion.chunk".to_string(),
            created: chrono::Utc::now().timestamp(),
            model: model.clone(),
            choices: vec![OpenAIStreamChoice {
                index: 0,
                delta: OpenAIDelta {
                    role: Some("assistant".to_string()),
                    content: extract_text_from_chunk(&chunk),
                    tool_calls: extract_tool_calls_from_chunk(&chunk),
                },
                finish_reason: chunk.candidates
                    .first()
                    .and_then(|c| c.finish_reason.as_ref())
                    .map(map_finish_reason),
            }],
        };
        
        // Format as SSE
        let json = serde_json::to_string(&openai_chunk)?;
        Ok(Bytes::from(format!("data: {}\n\n", json)))
    })
}

Image Generation Model Routing

Size and Quality Mapping

Location: src-tauri/src/proxy/handlers/openai.rs

pub fn map_image_generation_params(
    size: Option<String>,
    quality: Option<String>,
) -> (String, String) {
    // Parse size (e.g., "1920x1080" → "16:9")
    let aspect_ratio = if let Some(size_str) = size {
        if let Some((w, h)) = parse_dimensions(&size_str) {
            calculate_aspect_ratio(w, h)
        } else {
            "1:1".to_string()  // Default
        }
    } else {
        "1:1".to_string()
    };
    
    // Map quality to resolution
    let resolution = match quality.as_deref() {
        Some("hd") => "4K",
        Some("medium") => "2K",
        _ => "1K",  // standard
    };
    
    (aspect_ratio, resolution.to_string())
}

fn calculate_aspect_ratio(width: u32, height: u32) -> String {
    let ratio = width as f64 / height as f64;
    
    match () {
        _ if (ratio - 21.0/9.0).abs() < 0.1 => "21:9",
        _ if (ratio - 16.0/9.0).abs() < 0.1 => "16:9",
        _ if (ratio - 9.0/16.0).abs() < 0.1 => "9:16",
        _ if (ratio - 4.0/3.0).abs() < 0.1 => "4:3",
        _ if (ratio - 3.0/4.0).abs() < 0.1 => "3:4",
        _ => "1:1",
    }.to_string()
}

Model Suffix Parsing

pub fn parse_image_model_suffix(model: &str) -> (Option<String>, Option<String>) {
    // Example: "gemini-3-pro-image-16-9-4k" → ("16:9", "4K")
    
    let parts: Vec<&str> = model.split('-').collect();
    let mut aspect_ratio = None;
    let mut resolution = None;
    
    // Look for aspect ratio (e.g., "16-9")
    for i in 0..parts.len()-1 {
        if let (Ok(w), Ok(h)) = (parts[i].parse::<u32>(), parts[i+1].parse::<u32>()) {
            aspect_ratio = Some(format!("{}:{}", w, h));
        }
    }
    
    // Look for resolution (e.g., "4k")
    if let Some(last) = parts.last() {
        if last.to_lowercase().ends_with('k') {
            resolution = Some(last.to_uppercase());
        }
    }
    
    (aspect_ratio, resolution)
}

Parameter Adaptation

Thinking Mode Conversion

Anthropic Thinking Level → Gemini Budget:

pub fn convert_thinking_level_to_budget(level: &str) -> Option<i64> {
    match level.to_uppercase().as_str() {
        "NONE" => Some(0),
        "LOW" => Some(4096),
        "MEDIUM" => Some(8192),
        "HIGH" => Some(24576),
        _ => None,
    }
}

Token Limit Enforcement

Location: src-tauri/src/proxy/token_manager.rs:500

// Extract model-specific limits from quota data
if let Some(limit) = model.get("max_output_tokens").and_then(|v| v.as_u64()) {
    model_limits.insert(name.to_string(), limit);
}

Clamping at request time:

let max_allowed = token.model_limits
    .get(&normalized_model)
    .copied()
    .unwrap_or(131072);  // Default

if request.max_tokens > max_allowed {
    request.max_tokens = max_allowed;
    tracing::warn!("Clamped max_tokens to {}", max_allowed);
}

Request Routing Decision Tree

Error Handling

Model Not Found

if !is_valid_model(target_model) {
    return Err(AppError::InvalidModel {
        model: target_model.to_string(),
        suggestion: suggest_similar_model(target_model),
    });
}

Fallback Chain

let final_model = custom_mapping.get(&requested_model)
    .or_else(|| forwarding_rules.get(&requested_model))
    .or_else(|| normalize_to_standard_id(&requested_model))
    .unwrap_or_else(|| requested_model.clone());

Summary

The Model Router provides:

Multi-protocol support - OpenAI, Anthropic, Gemini
Flexible mapping - Custom + dynamic forwarding
Protocol conversion - Bidirectional transformation
Parameter adaptation - Enforce model-specific limits
Streaming support - SSE conversion for all protocols

Related Documentation:

System Overview - Full architecture
Proxy Server - HTTP server layer
Token Manager - Account selection

System Design

Advanced Topics

Overview

Key Responsibilities

Model Mapping System

Standard ID Normalization

Custom Mapping

Dynamic Forwarding Rules

Protocol Conversion

Request Flow

OpenAI → Gemini Conversion

Gemini → OpenAI Conversion

Claude → Gemini Conversion

Streaming Response Handling

SSE Stream Conversion

Image Generation Model Routing

Size and Quality Mapping

Model Suffix Parsing

Parameter Adaptation

Thinking Mode Conversion

Token Limit Enforcement

Request Routing Decision Tree

Error Handling

Model Not Found

Fallback Chain

Summary

Build docs developers (and LLMs) love

System Design

Advanced Topics

Documentation Index

​Overview

​Key Responsibilities

​Model Mapping System

​Standard ID Normalization

​Custom Mapping

​Dynamic Forwarding Rules

​Protocol Conversion

​Request Flow

​OpenAI → Gemini Conversion

​Gemini → OpenAI Conversion

​Claude → Gemini Conversion

​Streaming Response Handling

​SSE Stream Conversion

​Image Generation Model Routing

​Size and Quality Mapping

​Model Suffix Parsing

​Parameter Adaptation

​Thinking Mode Conversion

​Token Limit Enforcement

​Request Routing Decision Tree

​Error Handling

​Model Not Found

​Fallback Chain

​Summary

Build docs developers (and LLMs) love

Overview

Key Responsibilities

Model Mapping System

Standard ID Normalization

Custom Mapping

Dynamic Forwarding Rules

Protocol Conversion

Request Flow

OpenAI → Gemini Conversion

Gemini → OpenAI Conversion

Claude → Gemini Conversion

Streaming Response Handling

SSE Stream Conversion

Image Generation Model Routing

Size and Quality Mapping

Model Suffix Parsing

Parameter Adaptation

Thinking Mode Conversion

Token Limit Enforcement

Request Routing Decision Tree

Error Handling

Model Not Found

Fallback Chain

Summary