Skip to main content

Dynamic Model Configuration System

This document describes the new dynamic model configuration system that replaces static enums with flexible, runtime-configurable model definitions.

🎯 Overview​

The dynamic model system enables:

  • Runtime model discovery from external configuration sources
  • Automatic fallback to local configurations when external sources fail
  • Smart model resolution with fuzzy matching and aliases
  • Capability-based search to find models with specific features
  • Cost optimization by automatically selecting cheapest models for tasks

🏗️ Architecture​

Components​

  1. Model Configuration Server (scripts/modelServer.js)

    • Serves model configurations via REST API
    • Provides search and filtering capabilities
    • Can be hosted anywhere (GitHub, CDN, internal server)

  2. Dynamic Model Provider (src/lib/core/dynamicModels.ts)

    • Loads configurations from multiple sources with fallback
    • Caches configurations to reduce network requests
    • Validates configurations using Zod schemas
    • Provides intelligent model resolution

  3. Model Configuration (config/models.json)

    • JSON-based model definitions
    • Includes pricing, capabilities, and metadata
    • Supports aliases and provider defaults

🚀 Quick Start​

1. Environment Setup​

Before using the dynamic model system, ensure your provider configurations are set up correctly. See the Provider Configuration Guide for detailed instructions.

2. Start the Model Server​

# Start the configuration server
npm run model-server

# Or manually
node scripts/modelServer.js

Server runs on http://localhost:3001 by default.

2. Test the System​

# Run comprehensive tests
npm run test:dynamicModels

# Or manually
node test-dynamicModels.js

3. Use in Code​

// Preferred: import from the package export (no deep relative path)
import { dynamicModelProvider } from "@juspay/neurolink";
// Or, when importing within this repo's source (TypeScript):
// import { dynamicModelProvider } from "./src/lib/core/dynamicModels";

// Initialize the provider
await dynamicModelProvider.initialize();

// Resolve a model
const model = dynamicModelProvider.resolveModel("anthropic", "claude-3-opus");

// Search by capability
const visionModels = dynamicModelProvider.searchByCapability("vision");

// Get best model for use case
const bestCodingModel = dynamicModelProvider.getBestModelFor("coding");

📡 API Endpoints​

Model Server Endpoints​

  • GET /health - Health check
  • GET /api/v1/models - Get all model configurations
  • GET /api/v1/models/:provider - Get models for specific provider
  • GET /api/v1/search?capability=X&maxPrice=Y - Search models by criteria

Example API Usage​

# Get all models
curl http://localhost:3001/api/v1/models

# Get OpenAI models
curl http://localhost:3001/api/v1/models/openai

# Search for functionCalling models under $0.001
curl "http://localhost:3001/api/v1/search?capability=functionCalling&maxPrice=0.001"

🔧 Configuration Schema​

Model Configuration Structure​

{
  "version": "1.0.0",
  "lastUpdated": "2025-06-18T12:00:00Z",
  "models": {
    "anthropic": {
      "claude-3-opus": {
        "id": "claude-3-opus-20240229",
        "displayName": "Claude 3 Opus",
        "capabilities": ["functionCalling", "vision", "analysis"],
        "deprecated": false,
        "pricing": { "input": 0.015, "output": 0.075 },
        "contextWindow": 200000,
        "releaseDate": "2024-02-29"
      }
    }
  },
  "aliases": {
    "claude-latest": "anthropic/claude-3-opus",
    "best-coding": "anthropic/claude-3-opus"
  },
  "defaults": {
    "anthropic": "claude-3-sonnet"
  }
}

Key Fields​

  • id: Provider-specific model identifier
  • displayName: Human-readable model name
  • capabilities: Array of model capabilities (functionCalling, vision, etc.)
  • deprecated: Whether the model is deprecated
  • pricing: Input/output token costs per 1K tokens
  • contextWindow: Maximum context window size
  • releaseDate: Model release date

🎛️ Advanced Usage​

Configuration Sources​

The system tries multiple sources in order:

  1. process.env.MODEL_CONFIG_URL - Custom URL override
  2. http://localhost:3001/api/v1/models - Local development server
  3. https://raw.githubusercontent.com/juspay/neurolink/release/config/models.json - GitHub
  4. ./config/models.json - Local fallback

Model Resolution Logic​

// Exact match
resolveModel("anthropic", "claude-3-opus");

// Default model for provider
resolveModel("anthropic"); // Uses defaults.anthropic

// Alias resolution
resolveModel("anthropic", "claude-latest"); // Resolves alias

// Fuzzy matching
resolveModel("anthropic", "opus"); // Matches 'claude-3-opus'

Capability Search Options​

searchByCapability("functionCalling", {
  provider: "openai", // Filter by provider
  maxPrice: 0.001, // Maximum input price per 1K tokens
  excludeDeprecated: true, // Exclude deprecated models
});

🔄 Migration from Static Enums​

Before (Static Enums)​

export enum BedrockModels {
  CLAUDE_3_SONNET = "anthropic.claude-3-sonnet-20240229-v1:0",
  // Hard to maintain, becomes stale
}

After (Dynamic Resolution)​

// Backward compatible aliases
export const ModelAliases = {
  CLAUDE_LATEST: () =>
    dynamicModelProvider.resolveModel("anthropic", "claude-3"),
  GPT_LATEST: () => dynamicModelProvider.resolveModel("openai", "gpt-4"),
  BEST_CODING: () => dynamicModelProvider.getBestModelFor("coding"),
} as const;

// Usage stays the same
const provider = AIProviderFactory.createProvider(
  "anthropic",
  ModelAliases.CLAUDE_LATEST(),
);

🔐 Production Deployment​

Environment Variables​

# Custom model configuration URL
MODEL_CONFIG_URL=https://api.yourcompany.com/ai/models

# Server port (default: 3001)
MODEL_SERVER_PORT=8080

Hosting Configuration​

  1. GitHub Pages: Host models.json as static file
  2. CDN: Use CloudFlare/AWS CloudFront for global distribution
  3. Internal API: Integrate with existing infrastructure
  4. File System: Local configurations for air-gapped environments

Cache Strategy​

  • 5-minute cache: Balances freshness with performance
  • Graceful degradation: Falls back to cached data on network failures
  • Manual refresh: dynamicModelProvider.refresh() for immediate updates

🧪 Testing​

The test suite verifies:

✅ Model provider initialization ✅ Configuration loading from multiple sources ✅ Model resolution (exact, default, fuzzy, alias) ✅ Capability-based search ✅ Best model selection algorithms ✅ Error handling and fallbacks

Run tests with:

npm run test:dynamicModels

🚀 Benefits​

  • 🔄 Future-Proof: New models automatically available
  • 💰 Cost-Optimized: Runtime selection based on pricing
  • 🛡️ Reliable: Multiple fallback sources
  • ⚡ Fast: Cached configurations with smart invalidation
  • 🔒 Type-Safe: Zod schemas ensure runtime safety
  • 🔧 Backward Compatible: Existing code continues working

This system transforms static model definitions into a dynamic, self-updating platform that scales with the rapidly evolving AI landscape.