Skip to main content

​
Overview

The model_provider module provides a unified abstraction layer for working with both OpenAI models (GPT-5, GPT-4.1) and HuggingFace models (MediPhi, MedGemma) deployed via TGI or HuggingFace Inference Endpoints with OpenAI-compatible APIs.

​
ModelConfig

Configuration dataclass for language models.

​
Fields

​
name
str
required
Short name/key for the model (e.g., ‘gpt-4.1’, ‘mediphi’)
​
model_id
str
required
Full model identifier (e.g., ‘gpt-4.1’, ‘microsoft/MediPhi’)
​
provider
Literal['openai', 'huggingface']
required
Model provider type
​
endpoint_url_env
Optional[str]
Environment variable name for HuggingFace endpoint URL (e.g., ‘MEDIPHI_ENDPOINT_URL’). Only used for HuggingFace models. Should point to base endpoint URL without /v1.
​
temperature
float
default:"0.0"
Sampling temperature for the model

​
Example

from src.common.model_provider import ModelConfig

# OpenAI model configuration
openai_config = ModelConfig(
    name="gpt-5",
    model_id="gpt-5",
    provider="openai",
    temperature=0.0
)

# HuggingFace model configuration
hf_config = ModelConfig(
    name="mediphi",
    model_id="microsoft/MediPhi-Instruct",
    provider="huggingface",
    endpoint_url_env="MEDIPHI_ENDPOINT_URL",
    temperature=0.0
)

​
create_llm()

Factory function to create a language model based on configuration.

​
Signature

def create_llm(config: ModelConfig) -> BaseChatModel

​
Parameters

​
config
ModelConfig
required
ModelConfig instance specifying the model to create

​
Returns

​
BaseChatModel
BaseChatModel
Configured language model instance. OpenAI models return standard ChatOpenAI instances. HuggingFace models return ChatOpenAI instances pointing to a TGI/HF Inference Endpoint that exposes the OpenAI-compatible API.

​
Raises

  • ValueError: If endpoint URL environment variable is not set for HuggingFace models
  • ValueError: If HF_TOKEN environment variable is not set for HuggingFace models
  • ValueError: If provider is unknown

​
Behavior

OpenAI Models:
  • Returns standard ChatOpenAI instance
  • Uses model_id and temperature from config
  • Requires OPENAI_API_KEY environment variable
HuggingFace Models:
  • Returns ChatOpenAI instance configured for TGI/HF Inference Endpoint
  • Normalizes endpoint URL to OpenAI-compatible format (ending in /v1)
  • Requires environment variable specified in endpoint_url_env
  • Requires HF_TOKEN environment variable for authentication
  • Sets max_tokens=512 to prevent infinite loops in TGI models

​
Example

from src.common.model_provider import create_llm, MODELS_REGISTRY

# Create OpenAI model
gpt5_config = MODELS_REGISTRY["gpt-5"]
gpt5_llm = create_llm(gpt5_config)

# Create HuggingFace model
mediphi_config = MODELS_REGISTRY["microsoft/MediPhi-Instruct"]
mediphi_llm = create_llm(mediphi_config)

# Use the model
response = gpt5_llm.invoke("Explain the pathophysiology of preeclampsia.")

​
get_model_identity()

Resolve provider and model identity from registry hints and runtime LLM instance. Keeps metadata and pricing lookups consistent across OpenAI and OpenAI-compatible providers.

​
Signature

def get_model_identity(
    model_name: Optional[str] = None,
    llm: Optional[BaseChatModel] = None
) -> Dict[str, str]

​
Parameters

​
model_name
Optional[str]
Requested model name to look up in registry
​
llm
Optional[BaseChatModel]
Runtime LLM instance to extract model information from

​
Returns

​
identity
Dict[str, str]
Dictionary containing:
  • provider: Provider type (“openai”, “huggingface”, or “unknown”)
  • model_id: Full model identifier
  • model_name: Short model name/key

​
Resolution Logic

  1. First checks if model_name matches a key in MODELS_REGISTRY
  2. Then checks if runtime model name from llm.model_name matches registry
  3. Then searches registry for matching model_id
  4. Falls back to inference:
    • If model ID contains ”/”, infers “huggingface”
    • Otherwise infers “openai”
    • Uses “unknown” if no information available

​
Example

from src.common.model_provider import get_model_identity, create_llm, MODELS_REGISTRY

# Get identity from model name
identity = get_model_identity(model_name="gpt-5")
# {"provider": "openai", "model_id": "gpt-5", "model_name": "gpt-5"}

# Get identity from LLM instance
llm = create_llm(MODELS_REGISTRY["microsoft/MediPhi-Instruct"])
identity = get_model_identity(llm=llm)
# {"provider": "huggingface", "model_id": "microsoft/MediPhi-Instruct", "model_name": "mediphi"}

​
MODELS_REGISTRY

Canonical registry of all available models with their configurations.

​
Type

MODELS_REGISTRY: Dict[str, ModelConfig]

​
Registered Models

OpenAI Models:
  • gpt-5: GPT-5 model with temperature 0.0
  • gpt-5.2: GPT-5.2 model with temperature 0.0
HuggingFace Models:
  • microsoft/MediPhi-Instruct: Medical-specialized Phi model
    • Short name: mediphi
    • Endpoint: MEDIPHI_ENDPOINT_URL environment variable
  • google/medgemma-1.5-4b-it: Medical Gemma 1.5B instruction-tuned model
    • Short name: medgemma
    • Endpoint: MEDGEMMA_ENDPOINT_URL environment variable

​
Example

from src.common.model_provider import MODELS_REGISTRY, create_llm

# List all available models
for model_key in MODELS_REGISTRY:
    print(f"Available: {model_key}")

# Create model from registry
config = MODELS_REGISTRY["gpt-5"]
llm = create_llm(config)

# Access configuration details
mediphi_config = MODELS_REGISTRY["microsoft/MediPhi-Instruct"]
print(f"Provider: {mediphi_config.provider}")  # "huggingface"
print(f"Short name: {mediphi_config.name}")    # "mediphi"
print(f"Endpoint env: {mediphi_config.endpoint_url_env}")  # "MEDIPHI_ENDPOINT_URL"

​
Environment Variables

​
Required for All Models

  • OPENAI_API_KEY: API key for OpenAI models (loaded from .env if not set)

​
Required for HuggingFace Models

  • HF_TOKEN: HuggingFace authentication token
  • Model-specific endpoint URLs:
    • MEDIPHI_ENDPOINT_URL: Endpoint URL for MediPhi model
    • MEDGEMMA_ENDPOINT_URL: Endpoint URL for MedGemma model

​
Optional

  • PRICING_CONFIG_PATH: Path to custom pricing configuration JSON

​
Utility Functions

​
load_dotenv_if_needed()

Automatically loads environment variables from .env file if OPENAI_API_KEY is not already set. 
def load_dotenv_if_needed() -> None
Called automatically by create_llm(). Looks for .env file in project root (two levels up from the module).

​
URL Normalization

The module includes internal URL normalization to ensure HuggingFace endpoint URLs are properly formatted: 
# All of these are normalized to: https://host/v1
https://host
https://host/
https://host/v1
https://host/v1/
https://host/v1/chat/completions
This ensures consistent OpenAI-compatible API access regardless of how the endpoint URL is specified.

Build docs developers (and LLMs) love

Get started for freeTalk to us