Context Compaction

Overview

NeuroLink's Context Compaction system automatically manages conversation context windows, preventing overflow errors and maintaining conversation quality as sessions grow longer. It runs transparently before every generate() and stream() call.

Before each LLM call, the Budget Checker estimates the total input tokens needed (system prompt + conversation history + current prompt + tool definitions + file attachments) and compares them against the model's available context window. When usage exceeds the configured threshold (default: 80%), the ContextCompactor runs a 5-stage reduction pipeline:

Relevance Drop — Ask a decision model which earlier messages the current request still needs (needs a decision provider; skipped without one)
Tool Output Pruning — Replace old tool results with placeholders (cheapest, no LLM call)
File Read Deduplication — Keep only the latest read of each file (cheap, no LLM call)
LLM Summarization — Structured 10-section summary of older messages (expensive, requires LLM call)
Sliding Window Truncation — Remove oldest messages while preserving the first exchange (fallback, no LLM call)

If a provider still returns a context overflow error after compaction, the system detects it across all supported providers and retries with aggressive compaction.

Quick Start

import { NeuroLink } from "@juspay/neurolink";

const neurolink = new NeuroLink({
  conversationMemory: {
    enabled: true,
    enableSummarization: true,
    // Context compaction is enabled automatically when summarization is on.
    // All defaults work out of the box.
  },
});

That's it. Auto-compaction triggers at 80% context usage with every stage enabled.

The threshold is 80% by default and can be lowered per request by the context-budget decision — never raised. See that page for why the asymmetry is a hard invariant rather than a tuning choice.

SDK Configuration

The full contextCompaction block lives inside conversationMemory:

const neurolink = new NeuroLink({
  conversationMemory: {
    enabled: true,
    enableSummarization: true,
    summarizationProvider: "vertex", // Provider for summarization LLM calls
    summarizationModel: "gemini-2.5-flash", // Model for summarization LLM calls
    replayToolSteps: "marker", // How stored tool steps are replayed on later turns — see below
    contextCompaction: {
      enabled: true, // Enable auto-compaction (default: true when summarization enabled)
      threshold: 0.8, // Compaction trigger threshold, 0.0–1.0 (default: 0.80)
      enablePruning: true, // Enable Stage 1: tool output pruning (default: true)
      enableDeduplication: true, // Enable Stage 2: file read deduplication (default: true)
      enableSlidingWindow: true, // Enable Stage 4: sliding window fallback (default: true)
      maxToolOutputBytes: 50 * 1024, // Tool output max size in bytes (default: 51200)
      maxToolOutputLines: 2000, // Tool output max lines (default: 2000)
      fileReadBudgetPercent: 0.6, // File read budget as fraction of remaining context (default: 0.60)
    },
  },
});

Field	Type	Default	Description
`enabled`	`boolean`	`true` (when summarization enabled)	Master switch for auto-compaction
`threshold`	`number`	`0.80`	Usage ratio (0.0–1.0) that triggers compaction
`enablePruning`	`boolean`	`true`	Enable Stage 1: tool output pruning
`enableDeduplication`	`boolean`	`true`	Enable Stage 2: file read deduplication
`enableSlidingWindow`	`boolean`	`true`	Enable Stage 4: sliding window truncation fallback
`maxToolOutputBytes`	`number`	`51200` (50 KB)	Maximum tool output size in bytes before truncation
`maxToolOutputLines`	`number`	`2000`	Maximum tool output lines before truncation
`fileReadBudgetPercent`	`number`	`0.60`	Fraction of remaining context allocated for file reads

Environment Variables

These environment variables configure conversation memory and summarization, which in turn affect compaction behavior:

Variable	Default	Description
`NEUROLINK_MEMORY_ENABLED`	`"false"`	Set to `"true"` to enable conversation memory
`NEUROLINK_SUMMARIZATION_ENABLED`	`"true"`	Set to `"false"` to disable summarization
`NEUROLINK_TOKEN_THRESHOLD`	auto (80% of model context)	Override token threshold for triggering summarization
`NEUROLINK_SUMMARIZATION_PROVIDER`	`"vertex"`	Provider for summarization LLM calls
`NEUROLINK_SUMMARIZATION_MODEL`	`"gemini-2.5-flash"`	Model for summarization LLM calls
`NEUROLINK_MEMORY_MAX_SESSIONS`	`50`	Maximum number of sessions to keep in memory

Source: src/lib/config/conversationMemory.ts

Replaying Tool Steps on Later Turns

Every tool call a turn makes is stored in the session as a tool_call / tool_result pair — on both the in-memory and Redis backends, and for generate() as well as stream(). replayToolSteps decides how those rows reach the model on the session's later turns:

Mode	What the model sees on the next turn
`"marker"`	Default. One compact line per call, folded into the assistant turn: `[called lookup_order → ok]`, `→ error` when the call threw, was rejected or returned an error-shaped result (`isError: true`, an `error` field, or a failing `status`), or `→ unknown` when the result was lost to compaction.
`"full"`	The real tool call and its output, as tool-call / tool-result turns in the provider's own wire shape. A failed call's output carries the error text.
`"off"`	Tool rows are dropped from the prompt; only the surface user / assistant text is replayed.

Set it on the instance, or per request (the request wins):

const neurolink = new NeuroLink({
  conversationMemory: { enabled: true, replayToolSteps: "full" },
});

await neurolink.generate({
  input: { text: "And which carrier?" },
  context: { sessionId },
  replayToolSteps: "marker", // this turn only
});

CLI loop sessions accept the same knob: /set replayToolSteps full.

What to know before choosing "full":

Only paired rows are replayed. A call whose result never arrived (or the reverse) is dropped, because Anthropic and OpenAI reject an orphaned tool call / result outright. When compaction or a history slice leaves a call without its result, the memory layer's pair repair fills the gap with a placeholder result: the marker then reads → unknown and "full" replays the placeholder text ([Tool result unavailable - conversation was compacted]), never a fabricated outcome. A result whose call was lost is dropped altogether — the model never made that call.
A request that declares no tools degrades "full" to "marker". With disableTools: true, or a toolFilter that removes every tool, Anthropic rejects a prompt that carries tool blocks without a tools field, so such a turn is sent with markers instead.
Steps stay sequential. generate() stores one batch per loop step and the rows carry stepIndex, so a lookup followed by a send built from its output replays as two steps, not one parallel call.
Tokens. Each replayed step costs its arguments and output on every later turn of the session. The replay reads the tool_result content as the memory layer returns it, so Stage 1 pruning and the Redis backend's sendToolPreview (maxToolOutputBytes / maxToolOutputLines) both apply before a result is replayed.
Prompt caching. On Anthropic the replayed blocks sit inside messages, ahead of the rolling history breakpoints, so a session that switches modes mid-way changes its cached prefix. Pick a mode per session, not per turn, when cache hits matter.
Native replay is unchanged. Claude-on-Vertex and Gemini already replay tool steps from the stored rows in their own wire shape and do not go through this option; it governs the direct Anthropic, OpenAI-compatible and SageMaker generate() / stream() paths. On Amazon Bedrock only a turn that carries images or files goes through the shared history builder (where "full" degrades to "marker", since that path cannot carry tool turns); a text-only Bedrock turn sends the current prompt without stored history, so no mode applies to it. SageMaker's stream() sends no tools at all, so tools, toolChoice and prepareStep apply to its generate() only.
Context budget. Tool rows are stored at full size, but the budget check and the summarization trigger size them the way the chosen mode sends them — one line per call under "marker", nothing under "off" — so the default mode no longer triggers summarization on tool output the model never receives. This covers every budget check on the generate() and stream() paths, including the post-compaction and post-truncation re-checks. It does not yet reach the compaction stages themselves: once compaction runs, Stage 1 pruning, Stage 3 summarization and Stage 4 truncation still measure tool rows at stored size, so under "marker" or "off" a stage may prune or summarize more history than the prompt would have carried. Threading the mode into the compactor is a follow-up.

CLI Flags

The loop command accepts compaction-specific flags:

# Set a custom compaction threshold (0.0–1.0)
neurolink loop --compact-threshold 0.70

# Disable automatic context compaction entirely
neurolink loop --disable-compaction

Flag	Type	Default	Description
`--compact-threshold`	`number`	`0.8`	Context compaction trigger threshold (0.0–1.0)
`--disable-compaction`	`boolean`	`false`	Disable automatic context compaction

Source: src/cli/factories/commandFactory.ts:1466-1475

Public API Methods

`getContextStats(sessionId, provider?, model?)`

Get context usage statistics for a session. Returns token counts, usage ratio, and whether compaction should trigger.

Signature:

async getContextStats(
  sessionId: string,
  provider?: string,
  model?: string,
): Promise<{
  estimatedInputTokens: number;
  availableInputTokens: number;
  usageRatio: number;
  shouldCompact: boolean;
  messageCount: number;
} | null>

Returns null if conversation memory is not enabled or the session has no messages. The provider defaults to "openai" if not specified.

Example:

const stats = await neurolink.getContextStats(
  "session-1",
  "anthropic",
  "claude-sonnet-4-20250514",
);
if (stats) {
  console.log(`Usage: ${(stats.usageRatio * 100).toFixed(0)}%`);
  console.log(
    `Tokens: ${stats.estimatedInputTokens} / ${stats.availableInputTokens}`,
  );
  console.log(`Messages: ${stats.messageCount}`);
  console.log(`Needs compaction: ${stats.shouldCompact}`);
}

Source: src/lib/neurolink.ts:6624-6661

`compactSession(sessionId, config?)`

Manually trigger context compaction for a session. Runs the full 5-stage pipeline. After compaction, tool pairs are automatically repaired via repairToolPairs().

Signature:

async compactSession(
  sessionId: string,
  config?: CompactionConfig,
): Promise<CompactionResult | null>

Returns null if conversation memory is not enabled or the session has no messages.

Example:

const result = await neurolink.compactSession("session-1", {
  enablePrune: true,
  enableDeduplicate: true,
  enableSummarize: true,
  enableTruncate: true,
  // Stage 0 has no switch: it runs when a decision caller is injected and is
  // skipped when one is not, so there is no configuration in which it can
  // change behaviour on its own.
  pruneProtectTokens: 40_000,
  summarizationProvider: "vertex",
  summarizationModel: "gemini-2.5-flash",
});

if (result?.compacted) {
  console.log(`Stages used: ${result.stagesUsed.join(", ")}`);
  console.log(`Tokens saved: ${result.tokensSaved}`);
  console.log(`Before: ${result.tokensBefore}, After: ${result.tokensAfter}`);
}

Source: src/lib/neurolink.ts:6591-6618

`needsCompaction(sessionId, provider?, model?)`

Synchronous check of whether a session needs compaction. Uses checkContextBudget() internally with the default 80% threshold.

Signature:

needsCompaction(
  sessionId: string,
  provider?: string,
  model?: string,
): boolean

Returns false if conversation memory is not enabled or the session doesn't exist. The provider defaults to "openai" if not specified.

Example:

if (
  neurolink.needsCompaction(
    "session-1",
    "anthropic",
    "claude-sonnet-4-20250514",
  )
) {
  const result = await neurolink.compactSession("session-1");
  console.log(`Saved ${result?.tokensSaved} tokens`);
}

Source: src/lib/neurolink.ts:6666-6692

Types Reference

`CompactionStage`

type CompactionStage = "prune" | "deduplicate" | "summarize" | "truncate";

`CompactionResult`

Returned by compactSession() and ContextCompactor.compact().

type CompactionResult = {
  compacted: boolean; // Whether any compaction was applied
  stagesUsed: CompactionStage[]; // Which stages were used (in order)
  tokensBefore: number; // Estimated tokens before compaction
  tokensAfter: number; // Estimated tokens after compaction
  tokensSaved: number; // tokensBefore - tokensAfter
  messages: ChatMessage[]; // The compacted message array
};

`CompactionConfig`

Optional configuration passed to compactSession() or the ContextCompactor constructor.

type CompactionConfig = {
  enablePrune?: boolean; // Enable Stage 1 (default: true)
  enableDeduplicate?: boolean; // Enable Stage 2 (default: true)
  enableSummarize?: boolean; // Enable Stage 3 (default: true)
  enableTruncate?: boolean; // Enable Stage 4 (default: true)
  pruneProtectTokens?: number; // Recent tool output tokens to protect (default: 40,000)
  pruneMinimumSavings?: number; // Minimum tokens saved to declare pruning success (default: 20,000)
  pruneProtectedTools?: string[]; // Tool names that are never pruned (default: ["skill"])
  summarizationProvider?: string; // Provider for summarization LLM (default: "vertex")
  summarizationModel?: string; // Model for summarization LLM (default: "gemini-2.5-flash")
  keepRecentRatio?: number; // Fraction of messages to keep unsummarized (default: 0.3)
  truncationFraction?: number; // Fraction of oldest messages to remove in Stage 4 (default: 0.5)
  provider?: string; // Provider name for token estimation multipliers (default: "")
};

Source: src/lib/context/contextCompactor.ts:37-65

`BudgetCheckResult`

Returned by checkContextBudget().

type BudgetCheckResult = {
  withinBudget: boolean; // Whether the request fits within the context window
  estimatedInputTokens: number; // Estimated total input tokens
  availableInputTokens: number; // Available input tokens for this model
  usageRatio: number; // Usage ratio (0.0–1.0+)
  shouldCompact: boolean; // Whether auto-compaction should trigger
  breakdown: {
    systemPrompt: number; // Tokens from system prompt
    conversationHistory: number; // Tokens from conversation history
    currentPrompt: number; // Tokens from current user prompt
    toolDefinitions: number; // Tokens from tool definitions (content-based: JSON.stringify(tool).length / 4)
    fileAttachments: number; // Tokens from file attachments
  };
};

`BudgetCheckParams`

Parameters for checkContextBudget().

type BudgetCheckParams = {
  provider: string;
  model?: string;
  maxTokens?: number;
  systemPrompt?: string;
  conversationMessages?: Array<{ role: string; content: string }>;
  currentPrompt?: string;
  toolDefinitions?: unknown[];
  fileAttachments?: Array<{ content: string }>;
  compactionThreshold?: number; // 0.0–1.0, default: 0.80
};

Source: src/lib/context/budgetChecker.ts:18-54

The 5-Stage Pipeline

The ContextCompactor runs stages sequentially. Each stage only runs if the previous stage didn't bring tokens below the target budget.

#	Stage	`CompactionStage`	Needs a decision model
0	Relevance drop	`relevance`	yes — skipped entirely without one
1	Tool output pruning	`prune`	no
2	File read deduplication	`deduplicate`	no
3	LLM summarization	`summarize`	no (its gate uses one)
4	Sliding window truncation	`truncate`	no

result.stagesUsed reports the ones that actually ran, in order.

Stage 0: Relevance drop

File: src/lib/context/contextDecision.ts

Everything below Stage 0 is chronological: the pipeline's only notion of "droppable" is "old". Stage 0 is the one stage that asks what a message is for — one boolean per message ("is this needed to answer the current request?") in a single batch. On TypeSafe that costs the same for 200 messages as for one because decision latency is flat in question count; on Perplexity each further question adds about 65 ms and a request takes at most 128 (see its guide); on Cloudflare Clef a request takes at most 64 questions, which tryDecide() splits into batches of 64 (see its guide). NeuroLink also refuses a state it estimates at more than 16,000 tokens (about 60,000 characters of prose), so a longer set of eligible messages gets a refusal and the stage is skipped, as on any failure.

It is strictly additive. With no decision provider configured the stage does not run, stagesUsed omits relevance, and the pipeline behaves exactly as the four-stage one always did. It is also bounded by maxDropRatio and walks oldest-first, so when the cap binds it spares the newest candidates — the same recency assumption every other stage makes.

The summary gate

Stage 3 used to accept any non-empty string as a summary. When a decision model is configured, the generated summary is now checked first ("does this preserve every decision and open question?") and a rejected summary leaves the messages untouched so a later stage can try instead. The rejection is recorded on the span as compaction.stage3.summaryRejected, because a gate that silently discarded work would be indistinguishable from one that never ran.

Rejection is deliberately rare: the gate exists to catch a summary that lost a decision, not to second-guess wording.

Stage 1: Tool Output Pruning

File: src/lib/context/stages/toolOutputPruner.ts

Walks messages backwards, protecting the most recent tool outputs, and replaces older tool results with "[Tool result cleared]".

function pruneToolOutputs(
  messages: ChatMessage[],
  config?: PruneConfig,
): PruneResult;

PruneConfig:

Field	Type	Default	Description
`protectTokens`	`number`	`40,000`	Token budget of recent tool outputs to protect from pruning
`minimumSavings`	`number`	`20,000`	Minimum tokens that must be saved for pruning to be applied
`protectedTools`	`string[]`	`["skill"]`	Tool names that are never pruned
`provider`	`string`	—	Provider name for token estimation multiplier

PruneResult:

type PruneResult = {
  pruned: boolean; // Whether pruning was applied (savings >= minimumSavings)
  messages: ChatMessage[];
  tokensSaved: number;
};

Stage 2: File Read Deduplication

File: src/lib/context/stages/fileReadDeduplicator.ts

Detects multiple reads of the same file path. Keeps only the latest read, replaces earlier reads with "[File <path> - refer to latest read below]".

function deduplicateFileReads(messages: ChatMessage[]): DeduplicationResult;

DeduplicationResult:

type DeduplicationResult = {
  deduplicated: boolean; // Whether dedup was applied (requires 30%+ savings)
  messages: ChatMessage[];
  filesDeduped: number; // Number of unique files that had duplicates removed
};

A 30% savings threshold (DEDUP_THRESHOLD = 0.3) must be met for deduplication to be applied.

Stage 3: LLM Summarization

File: src/lib/context/stages/structuredSummarizer.ts

Uses the structured 10-section prompt to summarize older messages while keeping recent ones. Delegates to generateSummary() from the conversation memory system.

async function summarizeMessages(
  messages: ChatMessage[],
  config?: SummarizeConfig,
): Promise<SummarizeResult>;

SummarizeConfig:

Field	Type	Default	Description
`provider`	`string`	—	Provider for the summarization LLM call
`model`	`string`	—	Model for the summarization LLM call
`keepRecentRatio`	`number`	`0.3`	Fraction of messages to keep unsummarized (minimum: 4)
`memoryConfig`	`Partial<ConversationMemoryConfig>`	—	Memory config passed to `generateSummary()`

SummarizeResult:

type SummarizeResult = {
  summarized: boolean;
  messages: ChatMessage[]; // [summaryMessage, ...recentMessages]
  summaryText?: string; // Raw summary text
};

Behavior:

Will not summarize if there are 4 or fewer messages
Keeps at least 4 recent messages (or keepRecentRatio of total, whichever is greater)
Finds and incorporates any previous summary message for iterative merging
Summary message is inserted as a system role message with metadata.isSummary = true
If summarization fails (LLM error), the pipeline silently falls through to Stage 4

Stage 4: Sliding Window Truncation

File: src/lib/context/stages/slidingWindowTruncator.ts

Non-destructive fallback that removes the oldest messages from the middle of the conversation while always preserving the first user-assistant pair.

function truncateWithSlidingWindow(
  messages: ChatMessage[],
  config?: TruncationConfig,
): TruncationResult;

TruncationConfig:

Field	Type	Default	Description
`fraction`	`number`	`0.5`	Fraction of messages (after first pair) to remove

TruncationResult:

type TruncationResult = {
  truncated: boolean;
  messages: ChatMessage[]; // [firstPair..., truncationMarker, ...keptMessages]
  messagesRemoved: number; // Always an even number (maintains role alternation)
};

Behavior:

Will not truncate if there are 4 or fewer messages
Always preserves the first 2 messages (first user-assistant pair)
Removes an even number of messages to maintain role alternation
Inserts a system role truncation marker: "[Earlier conversation history was truncated to fit within context limits]"

ChatMessage Compaction Fields

The ChatMessage type has five fields used for non-destructive context management:

type ChatMessage = {
  // ... standard fields ...

  condenseId?: string; // UUID identifying this condensation group
  condenseParent?: string; // Points to the summary that replaces this message
  truncationId?: string; // UUID identifying this truncation group
  truncationParent?: string; // Points to the truncation marker that hides this message
  isTruncationMarker?: boolean; // Marks this message as a truncation boundary marker
};

Field	Purpose
`condenseId`	Set on the summary message. Groups all messages that were condensed together.
`condenseParent`	Set on original messages. Points to the `condenseId` of their summary.
`truncationId`	Set on the truncation marker. Groups all messages hidden by this truncation.
`truncationParent`	Set on original messages. Points to the `truncationId` of their marker.
`isTruncationMarker`	`true` on the synthetic marker message inserted where messages were removed.

Messages with condenseParent or truncationParent are filtered out by getEffectiveHistory() but remain in storage for potential rewind.

Source: src/lib/types/conversation.ts:270-279

Non-Destructive History

File: src/lib/context/effectiveHistory.ts

Messages are tagged rather than deleted, allowing compaction to be unwound.

`getEffectiveHistory(messages)`

Returns only visible messages by filtering out those with condenseParent or truncationParent.

function getEffectiveHistory(messages: ChatMessage[]): ChatMessage[];

`tagForCondensation(messages, fromIndex, toIndex, condenseId)`

Tags messages in [fromIndex, toIndex) with a condenseParent pointing to condenseId.

function tagForCondensation(
  messages: ChatMessage[],
  fromIndex: number,
  toIndex: number,
  condenseId: string,
): ChatMessage[];

`tagForTruncation(messages, fromIndex, toIndex, truncationId)`

Tags messages in [fromIndex, toIndex) with a truncationParent pointing to truncationId.

function tagForTruncation(
  messages: ChatMessage[],
  fromIndex: number,
  toIndex: number,
  truncationId: string,
): ChatMessage[];

`removeCondensationTags(messages, condenseId)`

Removes condenseParent tags from messages matching condenseId, making them visible again. Also removes the summary message itself (matched by condenseId + metadata.isSummary).

function removeCondensationTags(
  messages: ChatMessage[],
  condenseId: string,
): ChatMessage[];

`removeTruncationTags(messages, truncationId)`

Removes truncationParent tags from messages matching truncationId, making them visible again. Also removes the truncation marker itself (matched by truncationId + isTruncationMarker).

function removeTruncationTags(
  messages: ChatMessage[],
  truncationId: string,
): ChatMessage[];

Token Estimation

File: src/lib/utils/tokenEstimation.ts

Character-based token estimation with per-provider adjustment multipliers. Uses the same approach as Continue (GPT-tokenizer baseline + provider multipliers) without requiring a tokenizer dependency.

Constants

Constant	Value	Description
`CHARS_PER_TOKEN`	`4`	Characters per token for English text
`CODE_CHARS_PER_TOKEN`	`3`	Characters per token for code
`TOKEN_SAFETY_MARGIN`	`1.15`	Safety margin multiplier to avoid underestimation
`TOKENS_PER_MESSAGE`	`4`	Message framing overhead in tokens (role + delimiters)
`TOKENS_PER_CONVERSATION`	`24`	Conversation-level overhead in tokens
`IMAGE_TOKEN_ESTIMATE`	`1024`	Flat token estimate for images

Provider Multipliers

Applied on top of the base character estimate:

Provider	Multiplier	Notes
`anthropic`	`1.23`	Anthropic tokenizer produces ~23% more tokens
`google-ai`	`1.18`	Google AI Studio
`vertex`	`1.18`	Google Vertex AI
`mistral`	`1.26`	Mistral / Codestral
`openai`	`1.0`	Baseline (GPT-style)
`azure`	`1.0`	Same tokenizer as OpenAI
`bedrock`	`1.23`	Mostly Anthropic models
`ollama`	`1.0`
`litellm`	`1.0`
`huggingface`	`1.0`
`sagemaker`	`1.0`

Functions

estimateTokens(text, provider?, isCode?)

Estimate token count for a string.

function estimateTokens(
  text: string,
  provider?: string,
  isCode?: boolean,
): number;

Formula: ceil(text.length / charsPerToken) * providerMultiplier * TOKEN_SAFETY_MARGIN

estimateMessagesTokens(messages, provider?)

Estimate total token count for an array of messages, including per-message overhead and conversation-level overhead.

function estimateMessagesTokens(
  messages: Array<ChatMessage | { role: string; content: string }>,
  provider?: string,
): number;

truncateToTokenBudget(text, maxTokens, provider?)

Truncate text to fit within a token budget. Tries to cut at sentence or word boundaries. Appends "..." if truncated.

function truncateToTokenBudget(
  text: string,
  maxTokens: number,
  provider?: string,
): { text: string; truncated: boolean };

Context Window Registry

File: src/lib/constants/contextWindows.ts

Constants

Constant	Value	Description
`DEFAULT_CONTEXT_WINDOW`	`128,000`	Fallback when provider/model is unknown
`MAX_DEFAULT_OUTPUT_RESERVE`	`64,000`	Maximum output reserve when maxTokens not set
`DEFAULT_OUTPUT_RESERVE_RATIO`	`0.35`	Default output reserve as fraction of context

Functions

getContextWindowSize(provider, model?)

Resolve context window size. Priority: exact model match > provider _default > global DEFAULT_CONTEXT_WINDOW. Also supports partial model name prefix matching.

function getContextWindowSize(provider: string, model?: string): number;

getAvailableInputTokens(provider, model?, maxTokens?)

Calculate available input tokens: contextWindow - outputReserve.

function getAvailableInputTokens(
  provider: string,
  model?: string,
  maxTokens?: number,
): number;

getOutputReserve(contextWindow, maxTokens?)

Calculate output token reserve. Uses explicit maxTokens if provided, otherwise min(MAX_DEFAULT_OUTPUT_RESERVE, contextWindow * DEFAULT_OUTPUT_RESERVE_RATIO).

function getOutputReserve(contextWindow: number, maxTokens?: number): number;

`MODEL_CONTEXT_WINDOWS`

Complete per-provider, per-model context window registry:

Provider	Model	Context Window
anthropic	`_default`	200,000
	`claude-opus-4-20250514`	200,000
	`claude-sonnet-4-20250514`	200,000
	`claude-3-7-sonnet-20250219`	200,000
	`claude-3-5-sonnet-20241022`	200,000
	`claude-3-5-haiku-20241022`	200,000
	`claude-3-opus-20240229`	200,000
	`claude-3-sonnet-20240229`	200,000
	`claude-3-haiku-20240307`	200,000
openai	`_default`	128,000
	`gpt-4o`	128,000
	`gpt-4o-mini`	128,000
	`gpt-4-turbo`	128,000
	`gpt-4`	8,192
	`gpt-3.5-turbo`	16,385
	`o1`	200,000
	`o1-mini`	128,000
	`o1-pro`	200,000
	`o3`	200,000
	`o3-mini`	200,000
	`o4-mini`	200,000
	`gpt-4.1`	1,047,576
	`gpt-4.1-mini`	1,047,576
	`gpt-4.1-nano`	1,047,576
	`gpt-5`	1,047,576
google-ai	`_default`	1,048,576
	`gemini-2.5-pro`	1,048,576
	`gemini-2.5-flash`	1,048,576
	`gemini-2.0-flash`	1,048,576
	`gemini-1.5-pro`	2,097,152
	`gemini-1.5-flash`	1,048,576
	`gemini-3-flash-preview`	1,048,576
	`gemini-3-pro-preview`	1,048,576
vertex	`_default`	1,048,576
	`gemini-2.5-pro`	1,048,576
	`gemini-2.5-flash`	1,048,576
	`gemini-2.0-flash`	1,048,576
	`gemini-1.5-pro`	2,097,152
	`gemini-1.5-flash`	1,048,576
bedrock	`_default`	200,000
	`anthropic.claude-3-5-sonnet-20241022-v2:0`	200,000
	`anthropic.claude-3-5-haiku-20241022-v1:0`	200,000
	`anthropic.claude-3-opus-20240229-v1:0`	200,000
	`anthropic.claude-3-sonnet-20240229-v1:0`	200,000
	`anthropic.claude-3-haiku-20240307-v1:0`	200,000
	`amazon.nova-pro-v1:0`	300,000
	`amazon.nova-lite-v1:0`	300,000
azure	`_default`	128,000
	`gpt-4o`	128,000
	`gpt-4o-mini`	128,000
	`gpt-4-turbo`	128,000
	`gpt-4`	8,192
mistral	`_default`	128,000
	`mistral-large-latest`	128,000
	`mistral-medium-latest`	32,000
	`mistral-small-latest`	128,000
	`codestral-latest`	256,000
ollama	`_default`	128,000
litellm	`_default`	128,000
huggingface	`_default`	32,000
sagemaker	`_default`	128,000

Error Detection

File: src/lib/context/errorDetection.ts

Cross-provider regex patterns to detect context window overflow errors.

`isContextOverflowError(error)`

Returns true if the error matches any known context overflow pattern.

function isContextOverflowError(error: unknown): boolean;

Accepts Error objects, strings, or objects with message/error properties. Also inspects error.cause for nested errors.

`getContextOverflowProvider(error)`

Identifies which provider produced the context overflow error.

function getContextOverflowProvider(error: unknown): string | null;

Returns the provider name string or null if no match.

Supported Provider Patterns

Provider	Error Patterns
`openai`	`"This model's maximum context length is"`, `"reduce the length of the messages"`
`azure`	`"content_length_exceeded"`
`google`	`"RESOURCE_EXHAUSTED"`, `"exceeds the maximum number of tokens"`, `"content is too long"`
`bedrock`	`"ValidationException.*token"`, `"Input is too long"`, `"exceeds the model's maximum"`
`mistral`	`"context length exceeded"`, `"maximum number of tokens"`
`openrouter`	`"context_length_exceeded"`
`anthropic`	`"prompt is too long"`, `"input is too long"`, `"too many tokens"`

Non-Retryable Error Handling

When isContextOverflowError() detects that an error is a context overflow, the MCP generation retry loop (performMCPGenerationRetries) breaks immediately instead of retrying up to 3 times. This prevents wasting API calls on errors that cannot succeed without compaction.

Additionally, errors with statusCode === 400 or isRetryable === false are treated as non-retryable and break the retry loop immediately.

Post-Failure Compaction Passthrough

When a generation call fails with a context overflow error and compaction is triggered, the compacted messages are passed through via options.conversationMessages to directProviderGeneration(), which uses them instead of re-fetching from memory. The compaction target is set to Math.floor(availableInputTokens * 0.7) (70% of available context) to leave headroom.

Tool Output Limits

File: src/lib/context/toolOutputLimits.ts

Truncates individual tool outputs that exceed size limits. Can optionally save the full output to disk.

Constants

Constant	Value	Description
`MAX_TOOL_OUTPUT_BYTES`	`51200` (50 KB)	Maximum tool output in bytes
`MAX_TOOL_OUTPUT_LINES`	`2000`	Maximum tool output lines

`truncateToolOutput(output, options?)`

function truncateToolOutput(
  output: string,
  options?: TruncateOptions,
): TruncateResult;

TruncateOptions:

type TruncateOptions = {
  maxBytes?: number; // Default: MAX_TOOL_OUTPUT_BYTES (51200)
  maxLines?: number; // Default: MAX_TOOL_OUTPUT_LINES (2000)
  direction?: "head" | "tail"; // Which end to keep (default: "tail")
  saveToDisk?: boolean; // Save full output to disk (default: false)
  saveDir?: string; // Directory for saved output (default: os.tmpdir()/neurolink-tool-output)
};

TruncateResult:

type TruncateResult = {
  content: string; // Truncated content with notice appended
  truncated: boolean; // Whether truncation was applied
  savedPath?: string; // Path to saved full output (if saveToDisk was true)
  originalSize: number; // Original size in bytes
};

When truncated, a notice is appended: [Output truncated from X bytes to Y bytes] (with optional saved path).

File Token Budget

File: src/lib/context/fileTokenBudget.ts

Calculates how much of the remaining context window can be used for file reads. Implements fast-path for small files and preview mode for very large files.

Constants

Constant	Value	Description
`FILE_READ_BUDGET_PERCENT`	`0.6`	60% of remaining context allocated for file reads
`FILE_FAST_PATH_SIZE`	`102400` (100 KB)	Files below this size skip budget validation
`FILE_PREVIEW_MODE_SIZE`	`5242880` (5 MB)	Files above this size get preview-only mode
`FILE_PREVIEW_CHARS`	`2000`	Default preview size in characters

`calculateFileTokenBudget(contextWindow, currentTokens, maxOutputTokens)`

Calculate available token budget for file reads.

function calculateFileTokenBudget(
  contextWindow: number,
  currentTokens: number,
  maxOutputTokens: number,
): number;

Formula: floor((contextWindow - currentTokens - maxOutputTokens) * FILE_READ_BUDGET_PERCENT)

Returns 0 if remaining tokens is zero or negative.

`enforceAggregateFileBudget(files, provider, model, maxTokens)`

File: src/lib/context/fileTokenBudget.ts

Enforces a total token budget across all file attachments in a single request. When the aggregate content of all files exceeds the available context budget, files are truncated proportionally or dropped to fit.

This prevents the scenario where multiple large file attachments (e.g., 5 files totaling 2.8 MB) overflow the context window on the very first message — before any conversation history exists to compact.

function enforceAggregateFileBudget(
  files: Array<{ content: string; path?: string }>,
  provider: string,
  model?: string,
  maxTokens?: number,
): Array<{ content: string; path?: string }>;

Called automatically by buildMultimodalMessagesArray() before the file processing loop.

`shouldTruncateFile(fileSize, budget)`

Determine how a file should be handled based on its size and the token budget.

function shouldTruncateFile(
  fileSize: number,
  budget: number,
): { shouldTruncate: boolean; maxChars?: number; previewMode?: boolean };

Decision logic:

fileSize > FILE_PREVIEW_MODE_SIZE (5MB) → preview mode (2000 chars)
fileSize < FILE_FAST_PATH_SIZE (100KB) → no truncation
Otherwise → estimate tokens at 4 chars/token, truncate if exceeds budget

Tool Pair Repair

File: src/lib/context/toolPairRepair.ts

After compaction, tool_call/tool_result pairs may become orphaned (one half removed while the other remains). repairToolPairs validates every pair and inserts synthetic placeholders where needed.

function repairToolPairs(messages: ChatMessage[]): RepairResult;

RepairResult:

type RepairResult = {
  repaired: boolean; // Whether any repairs were made
  messages: ChatMessage[]; // Repaired message array (or original if no repairs)
  orphanedCallsFixed: number; // Number of tool_calls that got synthetic results
  orphanedResultsFixed: number; // Number of tool_results that got synthetic calls
};

Behavior:

A tool_call without a following tool_result gets a synthetic result: "[Tool result unavailable - conversation was compacted]"
A tool_result without a preceding tool_call gets a synthetic call: "[Tool call for <tool> - conversation was compacted]"
Synthetic messages have metadata.truncated = true

This runs automatically after compactSession().

CLI Session Warnings

File: src/cli/loop/session.ts:300-354

In loop mode, the CLI checks context budget after each turn and displays warnings:

At >60% usage (informational, gray text):

  Context: 65% used

At >=80% usage (warning, yellow text — compaction threshold reached):

  Context usage: 83% of window (166,000 / 200,000 tokens)
  Auto-compaction will trigger to preserve conversation quality.

These warnings only appear when contextCompaction.enabled is true in the session config.

Provider Support

Summary table of default context windows by provider:

Provider	Default Context Window	Notable Models
Anthropic	200,000	All Claude 3/3.5/4 models
OpenAI	128,000	GPT-4o, o1/o3 (200K), GPT-4.1/GPT-5 (1M+)
Google AI	1,048,576	Gemini 2.x/3.x (1M), Gemini 1.5 Pro (2M)
Vertex	1,048,576	Gemini 2.x (1M), Gemini 1.5 Pro (2M)
Bedrock	200,000	Claude models (200K), Nova (300K)
Azure	128,000	GPT-4o, GPT-4-turbo; GPT-4 (8K)
Mistral	128,000	Large/Small (128K), Medium (32K), Codestral (256K)
Ollama	128,000	Configurable per model
LiteLLM	128,000	Passthrough to underlying provider
Hugging Face	32,000	Model-dependent
SageMaker	128,000	Model-dependent

Overview​

Quick Start​

SDK Configuration​

Environment Variables​

Replaying Tool Steps on Later Turns​

CLI Flags​

Public API Methods​

getContextStats(sessionId, provider?, model?)​

compactSession(sessionId, config?)​

needsCompaction(sessionId, provider?, model?)​

Types Reference​

CompactionStage​

CompactionResult​

CompactionConfig​

BudgetCheckResult​

BudgetCheckParams​

The 5-Stage Pipeline​

Stage 0: Relevance drop​

The summary gate​

Stage 1: Tool Output Pruning​

Stage 2: File Read Deduplication​

Stage 3: LLM Summarization​

Stage 4: Sliding Window Truncation​

ChatMessage Compaction Fields​

Non-Destructive History​

getEffectiveHistory(messages)​

tagForCondensation(messages, fromIndex, toIndex, condenseId)​

tagForTruncation(messages, fromIndex, toIndex, truncationId)​

removeCondensationTags(messages, condenseId)​

removeTruncationTags(messages, truncationId)​

Token Estimation​

Constants​

Provider Multipliers​

Functions​

Context Window Registry​

Constants​

Functions​

MODEL_CONTEXT_WINDOWS​

Error Detection​

isContextOverflowError(error)​

getContextOverflowProvider(error)​

Supported Provider Patterns​

Non-Retryable Error Handling​

Post-Failure Compaction Passthrough​

Tool Output Limits​

Constants​

truncateToolOutput(output, options?)​

File Token Budget​

Constants​

calculateFileTokenBudget(contextWindow, currentTokens, maxOutputTokens)​

enforceAggregateFileBudget(files, provider, model, maxTokens)​

shouldTruncateFile(fileSize, budget)​

Tool Pair Repair​

CLI Session Warnings​

Provider Support​

Overview

Quick Start

SDK Configuration

Environment Variables

Replaying Tool Steps on Later Turns

CLI Flags

Public API Methods

`getContextStats(sessionId, provider?, model?)`

`compactSession(sessionId, config?)`

`needsCompaction(sessionId, provider?, model?)`

Types Reference

`CompactionStage`

`CompactionResult`

`CompactionConfig`

`BudgetCheckResult`

`BudgetCheckParams`

The 5-Stage Pipeline

Stage 0: Relevance drop

The summary gate

Stage 1: Tool Output Pruning

Stage 2: File Read Deduplication

Stage 3: LLM Summarization

Stage 4: Sliding Window Truncation

ChatMessage Compaction Fields

Non-Destructive History

`getEffectiveHistory(messages)`

`tagForCondensation(messages, fromIndex, toIndex, condenseId)`

`tagForTruncation(messages, fromIndex, toIndex, truncationId)`

`removeCondensationTags(messages, condenseId)`

`removeTruncationTags(messages, truncationId)`

Token Estimation

Constants

Provider Multipliers

Functions

Context Window Registry

Constants

Functions

`MODEL_CONTEXT_WINDOWS`

Error Detection

`isContextOverflowError(error)`

`getContextOverflowProvider(error)`

Supported Provider Patterns

Non-Retryable Error Handling

Post-Failure Compaction Passthrough

Tool Output Limits

Constants

`truncateToolOutput(output, options?)`

File Token Budget

Constants

`calculateFileTokenBudget(contextWindow, currentTokens, maxOutputTokens)`

`enforceAggregateFileBudget(files, provider, model, maxTokens)`

`shouldTruncateFile(fileSize, budget)`

Tool Pair Repair

CLI Session Warnings

Provider Support