Services

Service classes for AI chat, text-to-speech, speech-to-text, translation, summarization, rewriting, and content generation.

AIService

Multi-provider AI service with unified interface for Chrome AI, OpenAI, and Ollama.

Methods

function

configure(config: AIConfig, tabId?: number): Promise<boolean>

Configure AI client with provider settings.Parameters:

config: Configuration object
config.provider: ‘chrome-ai’ | ‘openai’ | ‘ollama’
config.chromeAi: Chrome AI settings (temperature, topK, enableImageSupport, enableAudioSupport)
config.openai: OpenAI settings (apiKey, model, temperature, maxTokens)
config.ollama: Ollama settings (endpoint, model, temperature, maxTokens)
tabId: Tab ID (extension mode only)

Returns: Success status

function

isConfigured(tabId?: number): boolean

Check if service is configured and ready.

function

chat(
  messages: Message[],
  options?: ChatOptions,
  tabId?: number
): AsyncGenerator<string>

Stream chat completion responses.Parameters:

messages: Array of message objects with role and content
options: { systemPrompt?, temperature?, maxTokens?, images?, audios? }
tabId: Tab ID (extension mode only)

Yields: Text chunks as they arriveExample:

for await (const chunk of aiService.chat(messages)) {
  console.log(chunk);
}

function

abortRequest(tabId?: number): void

Abort ongoing chat request.

Supported Providers

Chrome AI (Gemini Nano)

On-device inference (no API key required)
Multi-modal: Images and audio support
Requirements: Chrome 138+ with flags enabled
Model: Gemini Nano (2B-4B parameters)

OpenAI

Cloud-based (API key required)
Models: GPT-4, GPT-4 Turbo, GPT-4o, GPT-3.5 Turbo
Multi-modal: Images support (GPT-4 Vision)
Audio: Via Whisper transcription

Ollama

Self-hosted (local server)
Models: Llama 2, Mistral, Mixtral, etc.
Multi-modal: Depends on model (LLaVA for images)
Endpoint: Default http://localhost:11434

TTSService

Multi-provider Text-to-Speech service with streaming generation and audio queue management.

Methods

function

configure(config: TTSConfig, tabId?: number): boolean

Configure TTS client with provider settings.Parameters:

config.provider: ‘kokoro’ | ‘openai’ | ‘openai-compatible’
config.kokoro: Kokoro settings (modelId, voice, speed, device)
config.openai: OpenAI TTS settings (apiKey, model, voice, speed)
config['openai-compatible']: Generic TTS API settings
tabId: Tab ID (extension mode only)

function

isConfigured(tabId?: number): boolean

Check if service is configured and ready.

function

generateChunkedSpeech(
  text: string,
  voice?: string,
  chunkSize?: number,
  minChunkSize?: number,
  sessionId?: string,
  tabId?: number
): Promise<string[]>

Generate TTS audio with intelligent text chunking.Parameters:

text: Text to synthesize
voice: Voice ID (overrides config)
chunkSize: Target chunk size in characters (default: 500)
minChunkSize: Minimum chunk size (default: 100)
sessionId: Session identifier for tracking
tabId: Tab ID (extension mode only)

Returns: Array of audio blob URLs

function

playAudioSequence(
  audioBlobUrls: string[],
  sessionId?: string
): Promise<void>

Play sequence of audio chunks with queue management.Parameters:

audioBlobUrls: Array of blob URLs to play
sessionId: Session identifier

function

stopPlayback(): void

Stop current TTS playback and clear queue.

function

initializeBVMDConverter(scene: Scene): void

Initialize lip-sync animation converter with Babylon.js scene.

function

setSpeakCallback(callback: (text: string, bvmdUrl: string) => void): void

Set callback for triggering speak animations with lip sync.

Supported Providers

Kokoro TTS (Local)

On-device synthesis using ONNX runtime
Voices: 30+ high-quality neural voices
Backends: WebGPU (fast) or WASM (compatible)
Languages: English, Japanese, Chinese, Korean, French, Spanish
Lip Sync: Automatic BVMD generation for 3D character

OpenAI TTS

Cloud-based (API key required)
Models: tts-1 (fast), tts-1-hd (high quality)
Voices: alloy, echo, fable, onyx, nova, shimmer
Speed: Adjustable 0.25x to 4.0x

OpenAI-Compatible TTS

Generic TTS API (Cartesia, ElevenLabs, etc.)
Custom endpoint and voice configuration
OpenAI SDK compatibility

STTService

Multi-provider Speech-to-Text service supporting one-shot transcription and continuous streaming.

Methods

function

configure(config: STTConfig, tabId?: number): boolean

Configure STT client with provider settings.Parameters:

config.provider: ‘chrome-ai-multimodal’ | ‘openai’ | ‘openai-compatible’
config['chrome-ai-multimodal']: Chrome AI settings
config.openai: OpenAI Whisper settings
config['openai-compatible']: Generic STT API settings

function

isConfigured(tabId?: number): boolean

Check if service is configured.

function

startRecording(): Promise<boolean>

Start recording audio from microphone.Returns: Success status

function

stopRecording(): Promise<void>

Stop recording and transcribe audio.

function

setTranscriptionCallback(callback: (text: string) => void): void

Set callback for transcription results.

function

setErrorCallback(callback: (error: Error) => void): void

Set callback for error handling.

TranslatorService

Multi-provider translation service with support for Chrome AI Translator API and LLM-based translation.

Methods

function

configure(config: TranslatorConfig, tabId?: number): Promise<boolean>

Configure translator with provider settings.Parameters:

config.provider: ‘chrome-ai’ | ‘openai’ | ‘ollama’

function

translateStreaming(
  text: string,
  sourceLang: string,
  targetLang: string,
  tabId?: number
): AsyncGenerator<string>

Stream translation results.Parameters:

text: Text to translate
sourceLang: Source language code (e.g., ‘en’)
targetLang: Target language code (e.g., ‘es’)
tabId: Tab ID (extension mode only)

Yields: Translated text chunks

function

checkAvailability(
  sourceLang: string,
  targetLang: string,
  tabId?: number
): Promise<'readily' | 'downloading' | 'downloadable' | 'unavailable'>

Check if language pair is available for translation.

function

abort(tabId?: number): void

Abort ongoing translation request.

SummarizerService

Multi-provider text summarization service supporting Chrome AI Summarizer API and LLM-based summarization.

Methods

function

configure(config: SummarizerConfig, tabId?: number): Promise<boolean>

Configure summarizer with provider settings.

function

summarizeStreaming(
  text: string,
  options?: SummarizerOptions,
  tabId?: number
): AsyncGenerator<string>

Stream summarization results.Parameters:

text: Text to summarize
options.type: ‘tldr’ | ‘headline’ | ‘key-points’ | ‘teaser’
options.format: ‘plain-text’ | ‘markdown’
options.length: ‘short’ | ‘medium’ | ‘long’
tabId: Tab ID (extension mode only)

Yields: Summary text chunks

function

checkAvailability(tabId?: number): Promise<string>

Check if summarization is available.

function

abort(tabId?: number): void

Abort ongoing summarization request.

RewriterService

Multi-provider text rewriting service with tone, format, and length adjustments.

Methods

function

configure(config: RewriterConfig, tabId?: number): Promise<boolean>

Configure rewriter with provider settings.

function

rewriteStreaming(
  text: string,
  options?: RewriterOptions,
  tabId?: number
): AsyncGenerator<string>

Stream rewriting results.Parameters:

text: Text to rewrite
options.tone: ‘as-is’ | ‘more-formal’ | ‘more-casual’ | ‘professional’ | ‘friendly’
options.format: ‘as-is’ | ‘plain-text’ | ‘markdown’
options.length: ‘as-is’ | ‘shorter’ | ‘longer’
options.context: Additional context for rewriting
tabId: Tab ID (extension mode only)

Yields: Rewritten text chunks

function

abort(tabId?: number): void

Abort ongoing rewrite request.

WriterService

Multi-provider content generation service for creating new text from prompts.

Methods

function

configure(config: WriterConfig, tabId?: number): Promise<boolean>

Configure writer with provider settings.

function

writeStreaming(
  prompt: string,
  options?: WriterOptions,
  tabId?: number
): AsyncGenerator<string>

Stream content generation results.Parameters:

prompt: Writing prompt
options.tone: ‘neutral’ | ‘formal’ | ‘casual’ | ‘professional’ | ‘friendly’
options.format: ‘plain-text’ | ‘markdown’
options.length: ‘short’ | ‘medium’ | ‘long’
options.context: Additional context
tabId: Tab ID (extension mode only)

Yields: Generated text chunks

function

abort(tabId?: number): void

Abort ongoing write request.

ChatHistoryService

Service for managing persistent chat history with message trees and media storage.

Methods

function

saveChat(chatData: ChatData): Promise<string>

Save a chat with all associated data.Parameters:

chatData.chatId: Chat ID (auto-generated if not provided)
chatData.chatService: ChatService instance (for tree structure)
chatData.messages: Flat message array (backward compatibility)
chatData.title: Chat title (auto-generated if not provided)
chatData.isTemp: Skip saving if true
chatData.metadata: Additional metadata

Returns: Chat ID

function

loadChat(chatId: string): Promise<ChatRecord>

Load a chat with all data and media.Returns: Chat record with messages and metadata

function

listChats(
  offset?: number,
  limit?: number
): Promise<ChatRecord[]>

List saved chats with pagination.Parameters:

offset: Skip first N chats (default: 0)
limit: Max chats to return (default: 20)

Returns: Array of chat records (sorted by updatedAt descending)

function

deleteChat(chatId: string): Promise<void>

Delete a chat and all associated media.

function

updateChatTitle(chatId: string, newTitle: string): Promise<void>

Update the title of a saved chat.

function

searchChats(query: string): Promise<ChatRecord[]>

Search chats by title or content.

Usage Examples

AI Chat with Streaming

import AIServiceProxy from './services/proxies/AIServiceProxy';

// Configure
await AIServiceProxy.configure({
  provider: 'chrome-ai',
  chromeAi: {
    temperature: 1.0,
    topK: 3,
    enableImageSupport: true,
  },
});

// Chat with streaming
const messages = [
  { role: 'user', content: 'Hello!' },
];

let fullResponse = '';
for await (const chunk of AIServiceProxy.chat(messages)) {
  fullResponse += chunk;
  console.log(chunk);
}

Text-to-Speech with Kokoro

import { TTSServiceProxy } from './services/proxies';

// Configure
TTSServiceProxy.configure({
  enabled: true,
  provider: 'kokoro',
  kokoro: {
    modelId: 'onnx-community/Kokoro-82M-v1.0-ONNX',
    voice: 'af_heart',
    speed: 1.0,
    device: 'auto',
  },
});

// Generate and play
const audioUrls = await TTSServiceProxy.generateChunkedSpeech(
  "Hello, how can I help you?",
  null,
  500,
  100,
  'session_1'
);

await TTSServiceProxy.playAudioSequence(audioUrls, 'session_1');

Speech-to-Text Recording

import { STTServiceProxy } from './services/proxies';

// Set up callbacks
STTServiceProxy.setTranscriptionCallback((text) => {
  console.log('Transcription:', text);
});

STTServiceProxy.setErrorCallback((error) => {
  console.error('STT Error:', error);
});

// Start recording
await STTServiceProxy.startRecording();

// Stop when ready
await STTServiceProxy.stopRecording();

Translation with Streaming

import { TranslatorServiceProxy } from './services/proxies';

// Configure
await TranslatorServiceProxy.configure({
  provider: 'chrome-ai',
});

// Translate with streaming
let translation = '';
for await (const chunk of TranslatorServiceProxy.translateStreaming(
  'Hello, world!',
  'en',
  'es'
)) {
  translation += chunk;
  console.log(chunk);
}
// Result: "¡Hola, mundo!"

Contributing

API Reference

Services Reference

Services

AIService

Methods

Supported Providers

TTSService

Methods

Supported Providers

STTService

Methods

TranslatorService

Methods

SummarizerService

Methods

RewriterService

Methods

WriterService

Methods

ChatHistoryService

Methods

Usage Examples

​Services

​AIService

​Methods

​Supported Providers

​TTSService

​Methods

​Supported Providers

​STTService

​Methods

​TranslatorService

​Methods

​SummarizerService

​Methods

​RewriterService

​Methods

​WriterService

​Methods

​ChatHistoryService

​Methods

​Usage Examples

Services

AIService

Methods

Supported Providers

TTSService

Methods

Supported Providers

STTService

Methods

TranslatorService

Methods

SummarizerService

Methods

RewriterService

Methods

WriterService

Methods

ChatHistoryService

Methods

Usage Examples