📊 voyage-4-large newest flagship; voyage-3.5 default; voyage-code-3 / voyage-code-4 for code
🌍 voyage-multilingual-2 + domain-tuned (finance, law)
🔑 API key from dash.voyageai.com/api-keys

Setup Guide →

Jina AI

Embeddings + reranking

📊 jina-embeddings-v3 multilingual flagship
🔄 jina-reranker-v2 for retrieval reranking
🔍 jina-colbert-v2 late-interaction retrieval
🔑 API key from jina.ai

🎨 Direct Image Generation

Specialised image-gen providers (in addition to Vertex Imagen / OpenAI DALL-E / Anthropic / Bedrock):

Stability AI

Stable Image Ultra/Core + SD 3.5 family

🎨 Stable Image Ultra (flagship), Core (fast), SD 3.5 Large/Large-Turbo/Medium
🖼️ PNG output, aspect-ratio + negative-prompt + seed support
🔑 API key from platform.stability.ai/account/keys
🔄 Aliases: stability-ai, sd

Setup Guide →

Ideogram

Strong typography + design-focused image generation

📝 V3 default; V2/V2-Turbo/V1 also supported
🎨 magic_prompt + style + aspect_ratio controls
🔑 API key from developer.ideogram.ai

Recraft

Vector / illustration-focused image generation

🎨 recraftv3 (raster), recraftv3-svg (vector), recraftv2
📐 OpenAI-compat shape + style + size controls
🔑 API token from recraft.ai/api

💻 Local Providers

Run models entirely on your own hardware — no API key or internet required for inference:

Ollama

Run open-source models locally with full privacy

🖥️ 100% local inference — no data leaves your machine
🦙 70+ models: Llama, Mistral, Qwen, DeepSeek, Gemma, Phi, CodeLlama
🌐 Native Ollama API and OpenAI-compatible mode
🆓 No API key required

Setup Guide →

LM Studio

Run any supported model locally with a GUI app

🖥️ Download and run models via the LM Studio desktop application
🔍 Auto-discovers the loaded model from /v1/models (no model name required)
🌐 OpenAI-compatible API at http://localhost:1234/v1 by default
🆓 No API key needed for local use (key optional for reverse-proxy setups)
🔄 Aliases: lmstudio, lms

Setup Guide →

llama.cpp

High-performance local inference via llama-server

⚡ Run GGUF models with llama-server at http://localhost:8080/v1 by default
🔍 Auto-discovers the loaded model from /v1/models
🛠️ Tool support requires --jinja flag when starting llama-server
🆓 No API key needed for local use (key optional for reverse-proxy setups)
🔄 Aliases: llama.cpp

Setup Guide →

🔌 Aggregators & Proxies

Access multiple providers through unified interfaces:

OpenRouter

300+ models from 60+ providers

🌐 Single API across many AI providers (Anthropic, OpenAI, Google, Meta, etc.)
⚡ Automatic failover and routing
💰 Competitive pricing with cost optimization
🎯 Zero lock-in - switch models instantly
📊 Usage tracking dashboard
🆓 Free models available

Setup Guide →

OpenAI Compatible

OpenRouter, vLLM, LocalAI, and more

🌐 100+ models through OpenRouter
💻 Local deployment with vLLM
🔓 Self-hosted with LocalAI
🔄 Drop-in OpenAI replacement

Setup Guide →

LiteLLM

100+ providers through proxy

🔄 Unified API for 100+ providers
📊 Load balancing and fallbacks
💰 Cost tracking
🎯 Model routing

Setup Guide →

🧠 Decision-Only Providers

The two providers that serve decide rather than generate/stream. Each returns typed, calibrated judgments and emits no text, so neither appears in generation fallback chains or the health sweep.

TypeSafe (Jev)

Typed, calibrated judgments instead of text

🎯 boolean / choice / score answers, each with a calibrated confidence
⚡ Latency flat in question count — 1 question ~393 ms, 400 questions ~465 ms
💰 ~~$0.00002 per decision (~~$0.042/M input, output billed at zero)
🔌 Two transports: TypeSafe direct, or the Vercel AI Gateway
🛡️ Fails open — with no key configured, every consumer behaves exactly as before
🔑 API key from console.typesafe.ai/keys
🔄 Aliases: jev, typesafe-ai

Setup Guide →

Laya

Open-weights decision provider — the same typed boolean/choice/score answers as Jev, from Convai Innovations' Apache-2.0 checkpoints, at a Laya server or LiteLLM proxy route you configure

🧭 Serves decide only; built-in features use it when its key and base URL are set and neither TYPESAFE_API_KEY nor AI_GATEWAY_API_KEY is
📏 Refuses more than ~768 tokens of state (320 on english) before any network call, since its encoders read only 1,024 (512)
🔌 No built-in endpoint: LAYA_BASE_URL (or credentials.laya.baseURL) names a Laya server or a LiteLLM pass-through route to one
🔑 LAYA_API_KEY is the key that endpoint accepts; on LiteLLM, the route must be in the key's Allowed Routes

Setup Guide →

🧩 Additional Catalog Providers

Every provider below is a Tier-2 catalog entry. Provider-specific behavioural quirks may apply, so check the provider's catalog file. The whole integration is one JSON file under src/lib/providers/catalog/. Each page is generated from that file, which is also what the CI onboarding gate reads.

API Route

Claude Sonnet 4.6

🤖 8 models; default claude-sonnet-4-6 (1M context)
🛠️ Native tool calling + structured output together
💳 Free tier available
✅ Roster verified 2026-09-17 (authenticated GET /v1/models)
🔑 API key from api-route.com
🔄 Aliases: apiroute

Setup Guide →

Baseten

GLM 5.3 Flash

🤖 16 models; default zai-org/GLM-5.3-Flash (1M context)
🛠️ Native tool calling + structured output together
💳 Free tier available
✅ Roster verified 2026-09-03 (authenticated GET /v1/models)
🔑 API key from app.baseten.co

Setup Guide →

Friendli

zai-org/GLM-5.3

🤖 7 models; default zai-org/GLM-5.3 (1M context)
🛠️ Native tool calling — not combined with structured output in one request (HTTP 422 if both are set)
💳 Free tier available (rate limits are tight — pace requests 20s+ apart)
✅ Roster verified 2026-09-06 (authenticated GET /serverless/v1/models)
🔑 API key from suite.friendli.ai

Setup Guide →

GMI Cloud

MiniMaxAI/MiniMax-M3

🤖 1 model; default MiniMaxAI/MiniMax-M3 (1M context)
🛠️ Native tool calling + structured output together
💳 Free tier available
✅ Roster verified 2026-09-03 (authenticated GET /v1/models)
🔑 API key from console.gmicloud.ai
🔄 Aliases: gmi-cloud

Setup Guide →

Inception Labs

Mercury 2, Inception's enterprise diffusion LLM (dLLM)

🤖 1 model; default mercury-2 (125K context)
🛠️ Native tool calling + structured output together
💳 Free tier available
✅ Roster verified 2026-09-03 (authenticated GET /v1/models)
🔑 API key from platform.inceptionlabs.ai/dashboard/api-keys
🔄 Aliases: inception, mercury

Setup Guide →

io.net Intelligence

Meta: Llama 3.3 70B Instruct

🤖 34 models; default meta-llama/Llama-3.3-70B-Instruct (125K context)
🛠️ Native tool calling + structured output together
💳 Free tier available
✅ Roster verified 2026-09-03 (authenticated GET /v1/models)
🔑 API key from ai.io.net
🔄 Aliases: io-net

Setup Guide →

Mancer

DeepSeek V4 Flash

🤖 10 models; default deepseek-v4-flash (1M context)
⚠️ No tool calling — text generation and structured output only
💳 Free tier available
✅ Roster verified 2026-09-03 (authenticated GET /oai/v1/models; full response retained as evidence/mancer-roster-authenticated.json in the campaign scratchpad and every catalog price/limit machine-checked against it (Mancer re-prices — gpt-oss-120b input moved 0.024 → 0.022 within the day))
🔑 API key from mancer.tech/dashboard
🔄 Aliases: mancer-tech

Setup Guide →

Morph

Morph v3 Large

🤖 2 models; default morph-v3-large (262K context)
⚠️ No tool calling — text generation and structured output only
💳 Free tier requires a card on file (rate-limited to 5 req/min until added)
✅ Roster verified 2026-09-05 (authenticated GET /v1/models)
🔑 API key from morphllm.com/dashboard
🔄 Aliases: morphllm

Setup Guide →

Novita AI

zai-org/GLM-5.3 Flash

🤖 5 models; default zai-org/glm-5.3-flash (128K context)
🛠️ Native tool calling + structured output together
💳 Free tier available
✅ Roster verified 2026-09-05 (authenticated GET /openai/v1/models)
🔑 API key from novita.ai/settings/key-management

Setup Guide →

Upstage

Solar Pro 4

🤖 10 models; default solar-pro4 (512K context)
🛠️ Native tool calling + structured output together
💳 Free tier available
✅ Roster verified 2026-09-03 (authenticated GET /v1/models)
🔑 API key from console.upstage.ai/api-keys
🔄 Aliases: solar

Setup Guide →

🎙️ Voice Providers

Synthesize speech, transcribe audio, or run live voice sessions. Voice providers are separate from LLM providers — they handle audio I/O rather than text generation.

Text-to-Speech (TTS)

OpenAI TTS

Highest-quality text-to-speech

🎙️ Voices: alloy, echo, fable, onyx, nova, shimmer
🎵 Models: tts-1 (fast) and tts-1-hd (high quality)
🎼 Formats: MP3, WAV, OGG, Opus
🔑 Auth: API Key (OPENAI_API_KEY)

Setup Guide →

ElevenLabs

Best multilingual and voice-cloning TTS

🌍 Supports 30+ languages with natural prosody
🎭 Custom voice cloning from short audio samples
🎼 Formats: MP3, WAV (raw PCM, surfaced as pcm16), Opus (Ogg container)
🔑 Auth: API Key (ELEVENLABS_API_KEY)

Setup Guide →

Google TTS

1M characters/month free tier

💰 Generous free tier for standard voices
🌍 380+ voices across 50+ languages
🎼 Formats: MP3, WAV, OGG
🔑 Auth: Service Account

Setup Guide →

Azure TTS

Enterprise TTS with full SSML support

🏢 Fine-grained prosody control via SSML
🌍 400+ neural voices, 140+ languages
🎼 Formats: MP3, WAV (PCM), Opus (Ogg container)
🔑 Auth: API Key + Region

Setup Guide →

Fish Audio

Low-cost TTS with 15s voice cloning

💰 ~80% cheaper than ElevenLabs
🎭 15-second reference audio → custom voice
🌍 14 languages
🎼 Formats: MP3, WAV, PCM16 (raw)
🔑 Auth: API Key (FISH_AUDIO_API_KEY)

Setup Guide →

Cartesia

Low-latency Sonic models — synchronous + streaming

⚡ Sub-second turnaround on the synchronous /tts/bytes endpoint
🌊 Separate WebSocket streaming flow via CartesiaStream (voice server)
🎭 Voice cloning via dashboard upload
🎼 Formats: MP3 (44.1 kHz), WAV (PCM s16le @ 44.1 kHz), PCM16 (raw @ 24 kHz)
🔑 Auth: API Key (CARTESIA_API_KEY)

Setup Guide →

Speech-to-Text (STT)

Whisper (OpenAI)

Highest transcription accuracy

🎯 Best-in-class accuracy on diverse audio
🌍 Multilingual with automatic language detection
🎼 Formats: WAV, MP3, M4A, FLAC, OGG, OPUS, WEBM, MP4, MPEG, MPGA
🔑 Auth: API Key (OPENAI_API_KEY)

Setup Guide →

Deepgram

Real-time streaming transcription via WebSocket

⚡ Sub-300 ms word-level results over WebSocket
🌊 REST batch and WebSocket streaming modes
🎼 Formats: WAV, MP3, OGG, FLAC
🔑 Auth: API Key (DEEPGRAM_API_KEY)

Setup Guide →

Google STT

125+ languages with speaker diarization

🌍 Best fit for existing Google Cloud users
👥 Speaker diarization and multi-channel audio
🎼 Formats: WAV, FLAC, MP3, OGG
🔑 Auth: API Key (GOOGLE_AI_API_KEY / GEMINI_API_KEY) or Service Account (GOOGLE_APPLICATION_CREDENTIALS)

Setup Guide →

Azure STT

Enterprise STT with custom model training

🏢 Batch transcription and custom model support
🔒 Compliance controls for regulated industries
🎼 Formats: WAV (PCM), Ogg/Opus — convert MP3 to WAV first
🔑 Auth: API Key + Region

Setup Guide →

Realtime Voice

Realtime providers maintain a persistent bidirectional WebSocket connection, enabling low-latency spoken conversation with the AI model.

OpenAI Realtime

Low-latency bidirectional voice over WebSocket

⚡ Full-duplex audio stream with GPT-4o
🎵 Voice activity detection (VAD) built-in
🎼 Formats: WAV, Opus
🔑 Auth: API Key (OPENAI_API_KEY)

Setup Guide →

Gemini Live

Google's native realtime voice API

⚡ Native multimodal realtime session with Gemini
🎵 Supports audio + video input simultaneously
🎼 Formats: WAV, Opus
🔑 Auth: API Key (GOOGLE_AI_API_KEY or GEMINI_API_KEY)

Setup Guide →

🎬 Video Generation

Image-to-video and text-to-video providers (use via output: { mode: "video" }):

Vertex Veo 3.1 (default) — --videoProvider vertex
Kling (PiAPI) — --videoProvider kling (details)
Runway (Gen-3 Alpha / Gen-4 Turbo) — --videoProvider runway
Replicate — Wan-Alpha + many others — --videoProvider replicate (guide)

See Video Generation feature page for the full SDK / CLI surface.

👤 Avatar / Lip-Sync Generation

Talking-head video synthesis from a portrait image + audio (use via output: { mode: "avatar" }):

D-ID — --avatarProvider d-id (text-driven via Microsoft voices, or audio-driven)
HeyGen — --avatarProvider heygen (HeyGen avatar catalog id required)
Replicate (MuseTalk) — --avatarProvider replicate or musetalk (guide)

See docs/provider-integration/21-adding-new-modality.md for the architectural pattern.

🎵 Music / Sound Generation

Music + sound-effect generation (use via output: { mode: "music" }):

Beatoven.ai — --musicProvider beatoven (royalty-free background music)
ElevenLabs Music — --musicProvider elevenlabs-music (short SFX / loops up to 22s; same ELEVENLABS_API_KEY as TTS)
Lyria 3 Pro (Google) — --musicProvider lyria
Replicate (MusicGen) — --musicProvider replicate or musicgen (guide)

Quick Comparison

Provider	Free Tier	Enterprise	GDPR	Latency	Best For
Anthropic	Limited	✅	✅	Low	Reasoning, coding, Claude
Hugging Face	✅	❌	✅	Medium	Open source, experimentation
Google AI	✅	✅	✅	Low	Free tier, Gemini
Mistral AI	❌	✅	✅	Low	EU compliance, cost
OpenRouter	✅	✅	Varies	Low	Multi-model, automatic failover
OpenAI Compatible	Varies	✅	Varies	Varies	Flexibility, local deployment
LiteLLM	❌	✅	Varies	Low	Multi-provider, unified API
Azure OpenAI	❌	✅	✅	Low	Enterprise, Microsoft ecosystem
Vertex AI	❌	✅	✅	Low	Enterprise, GCP ecosystem
AWS Bedrock	❌	✅	✅	Low	Enterprise, AWS ecosystem
DeepSeek	❌	✅	❌	Low	Cost-effective reasoning, R1 model
NVIDIA NIM	❌	✅	Varies	Low	NVIDIA-hosted or self-hosted LLMs
LM Studio	✅ (Local)	❌	✅	Varies	Local GUI model management
llama.cpp	✅ (Local)	❌	✅	Varies	High-performance local GGUF inference
OpenAI TTS	❌	✅	✅	Low	High-quality TTS (tts-1-hd)
ElevenLabs	❌	✅	Varies	Low	Multilingual TTS, voice cloning
Google TTS	✅	✅	✅	Low	Cost-effective TTS, 1M chars free
Azure TTS	❌	✅	✅	Low	Enterprise TTS, SSML support
Fish Audio	❌	✅	Varies	Low	Low-cost TTS, voice cloning, 14 langs
Cartesia	❌	✅	Varies	Low	Low-latency Sonic models
Whisper	❌	✅	✅	Low	Best STT accuracy
Deepgram	❌	✅	Varies	Low	Real-time STT streaming (WebSocket)
Google STT	❌	✅	✅	Low	STT for GCP users, 125+ languages
Azure STT	❌	✅	✅	Low	Enterprise STT, custom models
OpenAI Realtime	❌	✅	✅	Low	Realtime bidirectional voice
Gemini Live	❌	✅	✅	Low	Realtime voice + video (Gemini)

Setup Strategies

Strategy 1: Free Tier First (Recommended for Development)

SDK Usage
CLI Usage

const ai = new NeuroLink({
providers: [
{
name: 'google-ai',
priority: 1,
config: { apiKey: process.env.GOOGLE_AI_KEY },
quotas: { daily: 1500 }
},
{
name: 'openai',
priority: 2,
config: { apiKey: process.env.OPENAI_API_KEY }
}
],
failoverConfig: { enabled: true, fallbackOnQuota: true }
});

    const result = await ai.generate({
      input: { text: "Hello world" }
    });

# Set up environment variables
export GOOGLE_AI_KEY="your-key"
export OPENAI_API_KEY="your-key"

    # Use with automatic failover
    npx @juspay/neurolink generate "Hello world" \
      --provider google-ai

Strategy 2: Multi-Region Enterprise

const ai = new NeuroLink({
  providers: [
    {
      name: "azure-us",
      region: "us-east",
      config: {
        /* Azure US */
      },
    },
    {
      name: "azure-eu",
      region: "eu-west",
      config: {
        /* Azure EU */
      },
    },
    {
      name: "bedrock-us",
      region: "us-east",
      config: {
        /* Bedrock US */
      },
    },
  ],
  loadBalancing: "latency-based",
});

const ai = new NeuroLink({
  providers: [
    {
      name: "mistral",
      priority: 1,
      config: { apiKey: process.env.MISTRAL_API_KEY },
    },
    {
      name: "azure-eu",
      priority: 2,
      config: {
        /* Azure EU region */
      },
    },
  ],
  compliance: {
    framework: "GDPR",
    dataResidency: "EU",
  },
});

Next Steps

Choose a provider based on your requirements (free tier, compliance, region)
Follow the setup guide to get your API key
Configure NeuroLink with the provider
Test the integration with a simple request
Add failover for production reliability

Multi-Provider Failover - High availability patterns
Cost Optimization - Reduce costs by 80-95%
Compliance & Security - GDPR, SOC2, HIPAA
Load Balancing - Distribution strategies
Voice Providers Comparison - TTS, STT, and Realtime capability matrix
Voice Provider Selection - Choosing the right voice provider

🆓 Free Tier Providers​

🤖 Direct AI Providers​

🏢 Enterprise Providers​

🌍 Compliance-Focused​

🧑‍💻 Hosted Inference Providers​

Fireworks AI​

Perplexity​

Cloudflare Workers AI​

Cohere​

Baseten​

GMI Cloud​

Inception Labs​

io.net Intelligence​

Mancer​

Upstage​

API Route​

🔍 Embedding-Only Providers​

Jina AI​

🎨 Direct Image Generation​

Ideogram​

Recraft​

💻 Local Providers​

🔌 Aggregators & Proxies​

🧠 Decision-Only Providers​

🧩 Additional Catalog Providers​

🎙️ Voice Providers​

Text-to-Speech (TTS)​

Speech-to-Text (STT)​

Realtime Voice​

🎬 Video Generation​

👤 Avatar / Lip-Sync Generation​

🎵 Music / Sound Generation​

Quick Comparison​

Setup Strategies​

Strategy 1: Free Tier First (Recommended for Development)​

Strategy 2: Multi-Region Enterprise​

Strategy 3: GDPR Compliance​

Next Steps​

Related Documentation​

🆓 Free Tier Providers

🤖 Direct AI Providers

🏢 Enterprise Providers

🌍 Compliance-Focused

🧑‍💻 Hosted Inference Providers

Fireworks AI

Perplexity

Cloudflare Workers AI

Cohere

Baseten

GMI Cloud

Inception Labs

io.net Intelligence

Mancer

Upstage

API Route

🔍 Embedding-Only Providers

Jina AI

🎨 Direct Image Generation

Ideogram

Recraft

💻 Local Providers

🔌 Aggregators & Proxies

🧠 Decision-Only Providers

🧩 Additional Catalog Providers

🎙️ Voice Providers

Text-to-Speech (TTS)

Speech-to-Text (STT)

Realtime Voice

🎬 Video Generation

👤 Avatar / Lip-Sync Generation

🎵 Music / Sound Generation

Quick Comparison

Setup Strategies

Strategy 1: Free Tier First (Recommended for Development)

Strategy 2: Multi-Region Enterprise

Strategy 3: GDPR Compliance

Next Steps

Related Documentation