Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/developer51709/Niko/llms.txt

Use this file to discover all available pages before exploring further.

Niko’s AI chat system is built on the OpenAI chat completions API (/v1/chat/completions). Because that standard is widely adopted, any compatible provider can be used as a drop-in replacement by setting just two environment variables — no code changes required unless you also want to change the model name.
src/.env
AI_INTEGRATIONS_OPENAI_API_KEY="<your provider API key>"
AI_INTEGRATIONS_OPENAI_BASE_URL="<provider base URL>"   # omit for native OpenAI
If only AI_INTEGRATIONS_OPENAI_API_KEY is set and AI_INTEGRATIONS_OPENAI_BASE_URL is left blank, requests are routed to the official OpenAI endpoint automatically.
These providers bill per token used. Prices are per 1 million tokens and reflect publicly listed rates at the time of writing.
ProviderRecommended ModelInput (per 1M tokens)Output (per 1M tokens)Notes
OpenAIgpt-4o-mini$0.15$0.60Default; best reliability
OpenAIgpt-4o$2.50$10.00Higher quality, higher cost
Anthropicclaude-3-5-haiku$0.80$4.00Via compatible wrapper only
Googlegemini-1.5-flash$0.075$0.30Via compatible wrapper
Mistralmistral-small$0.20$0.60Direct OpenAI-compat endpoint
Coherecommand-r$0.15$0.60Via compatible wrapper

​
Free Providers

These providers offer free tiers with OpenAI-compatible endpoints. Set AI_INTEGRATIONS_OPENAI_API_KEY to their key and AI_INTEGRATIONS_OPENAI_BASE_URL to their base URL.
  1. OpenRouter (Recommended) The best starting point for free AI. OpenRouter aggregates dozens of models from different providers under a single API.
    • Base URL: https://openrouter.ai/api/v1
    • Free models: meta-llama/llama-3.1-8b-instruct:free, mistralai/mistral-7b-instruct:free, google/gemma-2-9b-it:free, and dozens more
    • Rate limit: ~20 req/min (free tier)
    • Why recommended: Largest selection of free models, reliable uptime, easy model-switching, no credit card required to start
    • Recommended model: meta-llama/llama-3.1-8b-instruct:free
  2. Groq Extremely fast inference powered by custom LPU hardware. A strong choice if response latency matters.
    • Base URL: https://api.groq.com/openai/v1
    • Free models: llama-3.1-8b-instant, llama-3.3-70b-versatile, mixtral-8x7b-32768, gemma2-9b-it
    • Rate limit: 30 req/min, 14,400 req/day (free)
    • Why #2: Extremely fast inference (LPU hardware), generous daily quota, good model quality
    • Recommended model: llama-3.1-8b-instant
  3. Together AI Good model variety with a small amount of free starting credit that goes a long way at Niko’s token usage level.
    • Base URL: https://api.together.xyz/v1
    • Free models: Several Llama 3 and Mistral variants (free tier with $1 starting credit, then pay-as-you-go)
    • Rate limit: Varies by model
    • Recommended model: meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo
  4. Mistral AI (free tier) First-party Mistral endpoint with predictable behaviour, though the free quota is tighter than Groq or OpenRouter.
    • Base URL: https://api.mistral.ai/v1
    • Free models: mistral-small-latest (limited free quota)
    • Rate limit: 1 req/s, 500K tokens/month (free)
    • Recommended model: mistral-small-latest
  5. Cerebras Very fast for an 8B model on the free tier, though model selection is limited.
    • Base URL: https://api.cerebras.ai/v1
    • Free models: llama3.1-8b
    • Rate limit: 30 req/min (free)
    • Recommended model: llama3.1-8b
  6. Hugging Face Inference API Huge model library via the serverless API, but cold starts and variable rate limits can make it unreliable for real-time Discord interactions without a Pro plan.
    • Base URL: https://api-inference.huggingface.co/v1
    • Free models: Many open-source models via the serverless API
    • Rate limit: Varies; can be slow at peak times
    • Recommended model: meta-llama/Meta-Llama-3-8B-Instruct

​
How to Switch Providers

1

Set your API key

In src/.env, set AI_INTEGRATIONS_OPENAI_API_KEY to the API key you received from your chosen provider.
AI_INTEGRATIONS_OPENAI_API_KEY="sk-your-provider-key-here"
2

Set the base URL

Set AI_INTEGRATIONS_OPENAI_BASE_URL to the provider’s base URL from the tables above. Leave it blank to use native OpenAI.
AI_INTEGRATIONS_OPENAI_BASE_URL="https://openrouter.ai/api/v1"
3

Update the model name

Open src/utils/ai/openai_client.py and find the generate_reply_openai function. Look for the model= argument inside the chat.completions.create(...) call and update it to a model your new provider supports:
create_kwargs = dict(
    model="meta-llama/llama-3.1-8b-instruct:free",  # ← change this
    ...
)
4

Restart the bot

Save your changes and restart Niko. The new provider will be used for all subsequent AI requests.
If you are using Replit, the built-in OpenAI integration sets both OPENAI_API_KEY and the base URL automatically when you connect it — you don’t need to touch Secrets at all for the default OpenAI setup.

​
Token Usage

Niko’s prompt pipeline is optimised to keep token counts low. The exact cost per message depends on which optional experiments are enabled in your server configuration.
ScenarioApprox. input tokensApprox. output tokens
Basic message, no experiments~400~80
Better Context enabled~550~80
AI Actions enabled~900~80
Both experiments enabled~1,050~80
At OpenAI gpt-4o-mini pricing (0.15/0.15 / 0.60 per 1M tokens), 1,000 messages cost roughly 0.11∗∗withnoexperimentsand∗∗ 0.11** with no experiments and **~0.17 with both experiments enabled. Free-tier providers bring this cost to $0.

Build docs developers (and LLMs) love

Get started for freeTalk to us