Error Handling & Retry Logic

Antigravity Manager implements intelligent error handling and automatic retry mechanisms to provide a seamless experience even when upstream APIs encounter issues.

Error Classification

Location: src-tauri/src/error.rs and src-tauri/src/proxy/mappers/error_classifier.rs

Error Types

pub enum AppError {
    Database(rusqlite::Error),
    Network(String, Option<u16>),  // message, status code
    Io(std::io::Error),
    OAuth(String),
    Config(String),
    Account(String),
    Unknown(String),
}

HTTP Status Classification

Status Code	Category	Retry Strategy	Example
401	Auth expired	Auto-refresh token → retry	Invalid grant
403	Forbidden	Mark account → switch	Validation blocked
404	Not found	Short retry → rotate	Model not available
429	Rate limit	Exponential backoff → rotate	Too many requests
500-503	Server error	Retry with backoff	Overloaded
4xx	Client error	No retry	Invalid request

Error Classifier

pub fn classify_stream_error(e: &impl Display) -> (&'static str, String, &'static str) {
    let error_str = e.to_string();
    
    // Rate limit detection
    if error_str.contains("429") || error_str.contains("quota") {
        return (
            "rate_limit_error",
            "请求过于频繁，系统将自动切换账号重试".to_string(),
            "error.rate_limit"
        );
    }
    
    // Network issues
    if error_str.contains("timeout") || error_str.contains("connection") {
        return (
            "network_error",
            "网络连接不稳定，请检查您的网络或代理设置".to_string(),
            "error.network"
        );
    }
    
    // Auth errors
    if error_str.contains("401") || error_str.contains("unauthorized") {
        return (
            "authentication_error",
            "账号授权已过期，系统将自动刷新".to_string(),
            "error.auth"
        );
    }
    
    // Fallback
    ("api_error", error_str, "error.unknown")
}

Automatic Retry Logic

Location: Throughout proxy handlers

429 Rate Limit Handling

When a 429 Too Many Requests is received:

Immediate account rotation:

if status == 429 {
    tracing::warn!("[429] Rate limited on account {}, rotating...", account_id);
    
    // Get next available account
    let next_token = token_manager.get_next_available_token()?;
    
    // Retry request immediately with new account
    return retry_with_account(request, next_token).await;
}

Smart cooldown:

// Soft lock the rate-limited account for 5 seconds
token_manager.set_account_cooldown(account_id, Duration::from_secs(5));

Quota refresh:

// Trigger background quota refresh to update status
tokio::spawn(async move {
    let _ = account::fetch_account_quota(account_id).await;
});

401 Token Expiry

When a 401 Unauthorized is received:

Silent token refresh:

if status == 401 {
    tracing::info!("[401] Token expired for {}, refreshing...", account_id);
    
    // Attempt to refresh access token
    match oauth::refresh_access_token(account_id).await {
        Ok(new_token) => {
            // Update account with new token
            account::update_account_token(account_id, &new_token)?;
            
            // Retry original request with refreshed token
            return retry_request(request, account_id).await;
        }
        Err(e) => {
            // Mark account as disabled if refresh fails
            account::mark_account_disabled(
                account_id,
                &format!("Token refresh failed: {}", e)
            )?;
            
            // Rotate to next account
            let next_token = token_manager.get_next_available_token()?;
            return retry_with_account(request, next_token).await;
        }
    }
}

Automatic re-enablement: When token refresh succeeds, account is automatically re-enabled.

403 Validation Block

When a 403 Forbidden is received:

Account marking:

if status == 403 {
    let validation_url = extract_validation_url(&response_body);
    
    account::mark_account_forbidden(
        account_id,
        validation_url,
        Utc::now().timestamp()
    )?;
    
    tracing::warn!(
        "[403] Account {} blocked. Validation URL: {:?}",
        account_id,
        validation_url
    );
}

Immediate rotation:

// Skip this account and try next
let next_token = token_manager.get_next_available_token()?;
return retry_with_account(request, next_token).await;

UI notification: Account details page shows validation link for user action.

404 Model Not Found

Specific to Google Cloud Code API phased rollouts:

if status == 404 && request_path.contains("generateCode") {
    tracing::warn!(
        "[404] Model not available on account {}, trying next account...",
        account_id
    );
    
    // Short cooldown (5 seconds vs 8 for other errors)
    token_manager.set_account_cooldown(account_id, Duration::from_secs(5));
    
    // Quick retry with 300ms delay
    tokio::time::sleep(Duration::from_millis(300)).await;
    
    let next_token = token_manager.get_next_available_token()?;
    return retry_with_account(request, next_token).await;
}

500/503 Server Errors

Exponential backoff for temporary server issues:

if status >= 500 && status < 600 {
    let mut retry_count = 0;
    let max_retries = 3;
    
    while retry_count < max_retries {
        let delay = Duration::from_millis(100 * 2_u64.pow(retry_count));
        tokio::time::sleep(delay).await;
        
        match retry_request(request, account_id).await {
            Ok(response) => return Ok(response),
            Err(e) if retry_count < max_retries - 1 => {
                retry_count += 1;
                tracing::warn!("[5xx] Retry {} failed: {}", retry_count, e);
                continue;
            }
            Err(e) => return Err(e),
        }
    }
}

Backoff schedule:

Retry 1: 100ms
Retry 2: 200ms
Retry 3: 400ms

Account Rotation Strategy

Location: src-tauri/src/proxy/token_manager.rs

Smart Account Selection

pub fn get_next_available_token(&self) -> Result<ProxyToken, String> {
    let accounts = self.accounts.read().unwrap();
    
    // Filter available accounts
    let available: Vec<_> = accounts.iter()
        .filter(|acc| {
            !acc.disabled &&
            !acc.proxy_disabled &&
            !acc.validation_blocked &&
            !self.is_in_cooldown(&acc.id)
        })
        .collect();
    
    if available.is_empty() {
        return Err("No available accounts".to_string());
    }
    
    // Tiered routing: prioritize high-reset accounts
    let best = available.iter()
        .max_by_key(|acc| {
            let quota_score = calculate_quota_score(acc);
            let reset_score = calculate_reset_score(acc);
            quota_score * 100 + reset_score
        })
        .unwrap();
    
    Ok(best.to_proxy_token())
}

Quota Score Calculation

fn calculate_quota_score(account: &Account) -> u32 {
    let quota = match &account.quota {
        Some(q) => q,
        None => return 0,
    };
    
    // Average remaining percentage across all models
    let total: i32 = quota.models.iter()
        .map(|m| m.percentage)
        .sum();
    
    (total / quota.models.len() as i32) as u32
}

fn calculate_reset_score(account: &Account) -> u32 {
    // Prefer accounts that reset more frequently
    // Ultra: reset every hour (score: 100)
    // Pro: reset every 8 hours (score: 50)
    // Free: reset every 24 hours (score: 10)
    
    match account.subscription_tier.as_deref() {
        Some("ultra") => 100,
        Some("pro") => 50,
        _ => 10,
    }
}

Cooldown Management

struct AccountCooldown {
    account_id: String,
    until: Instant,
}

impl TokenManager {
    pub fn set_account_cooldown(&self, account_id: &str, duration: Duration) {
        let mut cooldowns = self.cooldowns.write().unwrap();
        cooldowns.insert(
            account_id.to_string(),
            Instant::now() + duration
        );
    }
    
    pub fn is_in_cooldown(&self, account_id: &str) -> bool {
        let cooldowns = self.cooldowns.read().unwrap();
        
        if let Some(until) = cooldowns.get(account_id) {
            Instant::now() < *until
        } else {
            false
        }
    }
}

Self-Healing Features

Location: Changelog references in README.md

Automatic Quota Refresh

Background task refreshes quota every N minutes:

pub async fn auto_refresh_quotas() {
    let mut interval = tokio::time::interval(Duration::from_secs(300)); // 5 min
    
    loop {
        interval.tick().await;
        
        let accounts = account::list_accounts().unwrap_or_default();
        for acc in accounts {
            if acc.disabled || acc.proxy_disabled {
                continue;  // Skip disabled accounts
            }
            
            match account::fetch_account_quota(&acc.id).await {
                Ok(quota) => {
                    // Update quota in database
                    let _ = account::update_account_quota(&acc.id, quota);
                }
                Err(e) if e.contains("403") => {
                    // Mark as forbidden if quota check fails
                    let _ = account::mark_account_forbidden(
                        &acc.id,
                        None,
                        Utc::now().timestamp()
                    );
                }
                Err(_) => {
                    // Ignore transient errors
                }
            }
        }
    }
}

Project ID Recovery

When project ID is missing or invalid:

if project_id.is_empty() || project_id == "projects/" {
    tracing::warn!("[Project] Invalid project ID detected, fetching...");
    
    match oauth::fetch_project_id(&account.refresh_token).await {
        Ok(pid) => {
            account::update_project_id(&account.id, &pid)?;
            project_id = pid;
        }
        Err(_) => {
            // Fallback to verified stable project
            project_id = "bamboo-precept-lgxtn".to_string();
            tracing::info!("[Project] Using fallback project ID");
        }
    }
}

Thinking Signature Recovery

When thinking blocks fail due to missing signatures:

pub fn strip_all_thinking_blocks(contents: Vec<Value>) -> Vec<Value> {
    contents.into_iter().map(|mut msg| {
        if let Some(parts) = msg["parts"].as_array_mut() {
            parts.retain(|part| {
                // Remove all parts with thought=true or thoughtSignature
                !part.get("thought").and_then(|t| t.as_bool()).unwrap_or(false) &&
                !part.get("thoughtSignature").is_some()
            });
        }
        msg
    }).collect()
}

This is automatically applied when:

Tool history exists (from previous turns)
Retry attempt is detected
Signature validation fails

Account Index Auto-Repair

If account index becomes corrupted:

pub fn rebuild_account_index() -> Result<(), String> {
    tracing::warn!("[Index] Rebuilding account index...");
    
    let data_dir = get_data_dir();
    let accounts_dir = data_dir.join("accounts");
    
    let mut index = Vec::new();
    
    for entry in fs::read_dir(&accounts_dir).map_err(|e| e.to_string())? {
        let entry = entry.map_err(|e| e.to_string())?;
        let path = entry.path();
        
        if path.extension().and_then(|s| s.to_str()) == Some("json") {
            if let Ok(account) = load_account_from_file(&path) {
                index.push(account.id);
            }
        }
    }
    
    save_account_index(&index)?;
    tracing::info!("[Index] Rebuilt with {} accounts", index.len());
    
    Ok(())
}

Triggered automatically when:

Index file is missing
Index contains invalid entries
Account count mismatch detected

Quota Protection

Prevents requests when quota is exhausted:

pub fn has_available_quota(account: &Account, model: &str) -> bool {
    let quota = match &account.quota {
        Some(q) => q,
        None => return true,  // Allow if unknown
    };
    
    // Normalize model name for comparison
    let normalized = normalize_model_name(model);
    
    // Find matching quota entry
    for model_quota in &quota.models {
        if model_quota.name == normalized {
            return model_quota.percentage > 0;
        }
    }
    
    // Allow if no specific quota found
    true
}

Integrated into account selection:

let available: Vec<_> = accounts.iter()
    .filter(|acc| has_available_quota(acc, &requested_model))
    .collect();

Error Recovery Modes

Permissive Mode

For first-time thinking requests (no history):

if !has_thinking_history && is_thinking_enabled {
    tracing::info!(
        "[Thinking-Mode] First thinking request detected. Using permissive mode."
    );
    // Allow upstream to validate, don't enforce signature checks
}

Strict Mode

For tool calls with thinking:

if needs_signature_check && !has_valid_signature() {
    tracing::warn!(
        "[Thinking-Mode] No valid signature for function calls. Disabling thinking."
    );
    is_thinking_enabled = false;
}

Adaptive Mode

Dynamically adjusts based on context:

if adaptive_thinking_enabled {
    let budget = match request_complexity {
        Complexity::Low => 4096,
        Complexity::Medium => 8192,
        Complexity::High => 24576,
    };
    
    gen_config["thinkingConfig"]["thinkingBudget"] = json!(budget);
}

Monitoring & Logging

All errors are logged with context:

tracing::error!(
    "[Error] Request failed: status={}, account={}, model={}, error={}",
    status,
    account_id,
    model,
    error_message
);

Accessible via:

UI logs page (/api/logs)
System logs (stored in data directory)
Debug console (if enabled)

Best Practices

Add multiple accounts for seamless rotation
Enable auto-refresh to keep quota status current
Monitor logs for recurring errors
Respond to 403s by following validation links
Keep tokens fresh by using accounts regularly

Overview

Endpoints

Protocol Conversion

Error Handling & Retry

Error Handling & Retry Logic

Error Classification

Error Types

HTTP Status Classification

Error Classifier

Automatic Retry Logic

429 Rate Limit Handling

401 Token Expiry

403 Validation Block

404 Model Not Found

500/503 Server Errors

Account Rotation Strategy

Smart Account Selection

Quota Score Calculation

Cooldown Management

Self-Healing Features

Automatic Quota Refresh

Project ID Recovery

Thinking Signature Recovery

Account Index Auto-Repair

Quota Protection

Error Recovery Modes

Permissive Mode

Strict Mode

Adaptive Mode

Monitoring & Logging

Best Practices

See Also

Build docs developers (and LLMs) love

Overview

Endpoints

Protocol Conversion

Documentation Index

​Error Handling & Retry Logic

​Error Classification

​Error Types

​HTTP Status Classification

​Error Classifier

​Automatic Retry Logic

​429 Rate Limit Handling

​401 Token Expiry

​403 Validation Block

​404 Model Not Found

​500/503 Server Errors

​Account Rotation Strategy

​Smart Account Selection

​Quota Score Calculation

​Cooldown Management

​Self-Healing Features

​Automatic Quota Refresh

​Project ID Recovery

​Thinking Signature Recovery

​Account Index Auto-Repair

​Quota Protection

​Error Recovery Modes

​Permissive Mode

​Strict Mode

​Adaptive Mode

​Monitoring & Logging

​Best Practices

​See Also

Build docs developers (and LLMs) love

Error Handling & Retry Logic

Error Classification

Error Types

HTTP Status Classification

Error Classifier

Automatic Retry Logic

429 Rate Limit Handling

401 Token Expiry

403 Validation Block

404 Model Not Found

500/503 Server Errors

Account Rotation Strategy

Smart Account Selection

Quota Score Calculation

Cooldown Management

Self-Healing Features

Automatic Quota Refresh

Project ID Recovery

Thinking Signature Recovery

Account Index Auto-Repair

Quota Protection

Error Recovery Modes

Permissive Mode

Strict Mode

Adaptive Mode

Monitoring & Logging

Best Practices

See Also