Skip to main content

Analytics & Evaluation

Advanced analytics and AI response evaluation features for monitoring usage, performance, and quality.

🎯 Overview​

NeuroLink provides comprehensive analytics and evaluation capabilities to help you monitor AI usage, track performance, and assess response quality. These features are essential for production applications and enterprise deployments.

📊 Analytics Features​

Usage Analytics​

Track detailed metrics about your AI interactions:

import { NeuroLink } from "@juspay/neurolink";

const neurolink = new NeuroLink({
  analytics: {
    enabled: true,
    endpoint: "https://analytics.yourcompany.com",
    apiKey: process.env.ANALYTICS_API_KEY,
  },
});

// Analytics automatically tracked
const result = await neurolink.generate({
  input: { text: "Generate report" },
  context: {
    userId: "user123",
    sessionId: "sess456",
    department: "engineering",
  },
});

CLI Analytics​

Enable analytics in CLI commands:

# Enable analytics for single command
npx @juspay/neurolink gen "Analyze data" --enable-analytics

# With custom context
npx @juspay/neurolink gen "Business analysis" \
  --enable-analytics \
  --context '{"team":"product","project":"dashboard"}' \
  --debug

Tracked Metrics​

  • Usage Statistics: Request count, frequency, patterns
  • Performance Metrics: Response time, token usage, costs
  • Provider Statistics: Success rates, error patterns, latency
  • Cost Analysis: Per-provider costs, budget tracking
  • User Analytics: Usage by user, team, or department
  • Quality Metrics: Response evaluation scores

🔍 Response Evaluation​

AI-Powered Quality Assessment​

// Enable evaluation for quality scoring
const result = await neurolink.generate({
  input: { text: "Write production code" },
  enableEvaluation: true,
  evaluationDomain: "Senior Software Engineer",
  evaluationCriteria: ["accuracy", "completeness"],
});

console.log(result.evaluation);
// {
//   overall: 9.2,
//   relevance: 9.5,
//   accuracy: 9.0,
//   completeness: 8.8,
//   reasoning: "Code follows best practices...",
//   alertSeverity: "none"
// }

CLI Evaluation​

# Basic evaluation
npx @juspay/neurolink gen "Write API documentation" --enable-evaluation

# Domain-specific evaluation
npx @juspay/neurolink gen "Design system architecture" \
  --enable-evaluation \
  --evaluation-domain "Solutions Architect"

# Combined analytics and evaluation
npx @juspay/neurolink gen "Create test plan" \
  --enable-analytics \
  --enable-evaluation \
  --evaluation-domain "QA Engineer" \
  --debug

Evaluation Domains​

Specialized evaluation contexts:

  • Technical: Senior Software Engineer, DevOps Specialist, Data Scientist
  • Business: Product Manager, Business Analyst, Marketing Manager
  • Creative: Content Writer, UX Designer, Creative Director
  • Academic: Research Scientist, Technical Writer, Educator

📈 Analytics Collection​

Per-Request Analytics​

Analytics are collected on a per-request basis and included in each result:

// Enable analytics for a single request
const result = await neurolink.generate({
  input: { text: "Generate documentation" },
  enableAnalytics: true,
});

// Access analytics from the result
console.log(result.analytics);
// {
//   totalTokens: 1523,
//   promptTokens: 421,
//   completionTokens: 1102,
//   cost: 0.0045,
//   durationMs: 1456,
//   provider: "openai",
//   model: "gpt-4o"
// }

Middleware-Based Analytics​

For application-wide analytics collection, use the analytics middleware:

import { getAnalyticsMetrics, clearAnalyticsMetrics } from "@juspay/neurolink";

// Analytics are automatically collected by the middleware
const metrics = getAnalyticsMetrics();

// Process or export metrics as needed
console.log(metrics);

// Clear metrics after processing
clearAnalyticsMetrics();

🔧 Configuration​

Environment Variables​

# Evaluation Configuration
NEUROLINK_EVALUATION_PROVIDER="google-ai"
NEUROLINK_EVALUATION_MODEL="gemini-2.5-flash"
NEUROLINK_EVALUATION_THRESHOLD="7"

Per-Request Configuration​

Analytics and evaluation are configured on a per-request basis:

// Enable analytics and evaluation for specific requests
const result = await neurolink.generate({
  input: { text: "Your prompt" },
  enableAnalytics: true,
  enableEvaluation: true,
  evaluationDomain: "Senior Software Engineer",
  evaluationCriteria: ["accuracy", "completeness"],
});

📊 Available Methods​

The following methods are fully available in the SDK for advanced analytics, performance monitoring, and cost calculations:

MethodDescription
neurolink.getProviderMetrics()Get aggregated provider metrics and performance
neurolink.getCostAnalysis()Get granular cost breakdown and projections
neurolink.getTeamAnalytics()Get team-wide usage, unique users, and quality
neurolink.getProviderStatus()Get provider availability status
neurolink.getProviderHealthSummary()Get health summary for all providers
neurolink.getToolExecutionMetrics()Get tool execution statistics
getAnalyticsMetrics()Standalone middleware function for analytics data
import { NeuroLink } from "@juspay/neurolink";

const neurolink = new NeuroLink();

// Get provider metrics
const providerMetrics = await neurolink.getProviderMetrics({
  timeRange: "last_7_days",
});
console.log(providerMetrics);

// Get granular cost analysis
const costAnalysis = await neurolink.getCostAnalysis({
  includeProjections: true,
});
console.log(costAnalysis);

// Get team usage analytics
const teamAnalytics = await neurolink.getTeamAnalytics({
  teamId: "engineering-core",
});
console.log(teamAnalytics);

📊 Use Cases​

Performance Monitoring​

// Monitor provider performance
const perfMetrics = await neurolink.getProviderMetrics({
  providers: ["openai", "google", "anthropic"],
  timeRange: "last_24_hours",
});

// Identify best performing provider by average latency
const bestProvider = perfMetrics.providers.sort(
  (a, b) => a.averageLatency - b.averageLatency,
)[0];

console.log(`Best provider: ${bestProvider.name}`);

Cost Optimization​

// Track costs and project future spend
const costAnalysis = await neurolink.getCostAnalysis({
  timeRange: "current_month",
  groupBy: ["provider", "model"],
  includeProjections: true,
});

console.log(`Current spend: $${costAnalysis.totalCost}`);
console.log(`Projected next month: $${costAnalysis.projections?.nextMonth}`);

Quality Assurance​

# Batch evaluate responses for quality
cat prompts.txt | while read prompt; do
  npx @juspay/neurolink gen "$prompt" \
    --enable-evaluation \
    --evaluation-domain "Senior Engineer" \
    --json >> evaluations.json
done

# Analyze quality trends
jq '.evaluation.overall' evaluations.json | awk '{sum+=$1} END {print "Average quality:", sum/NR}'

🚀 Enterprise Features​

Team Analytics​

// Department-level analytics and user attribution
const teamMetrics = await neurolink.getTeamAnalytics({
  departments: ["engineering", "product", "marketing"],
  timeRange: "last_30_days",
});

console.log(`Total active unique users: ${teamMetrics.uniqueUsers}`);

Custom Metrics​

// Define custom analytics tracking
const result = await neurolink.generate({
  input: { text: "Generate report" },
  analytics: {
    customMetrics: {
      feature: "report_generation",
      complexity: "high",
      businessValue: "critical",
    },
  },
});

Compliance Monitoring​

# Audit trail with evaluation
npx @juspay/neurolink gen "Sensitive analysis" \
  --enable-analytics \
  --enable-evaluation \
  --context '{"compliance":"required","audit":"true"}' \
  --evaluation-domain "Compliance Officer"

📚 Related Documentation​