Memory & Context Architecture
NexusAI's Memory & Context engine enables long-horizon reasoning, stateful conversations, and dynamic knowledge retrieval. Learn how ephemeral buffers, vector stores, and episodic memory work together to give your agents true contextual awareness.
How Context Works
Traditional LLMs operate statelessly. NexusAI introduces a tiered memory system that persists state across sessions, retrieves relevant historical data via semantic search, and maintains strict token budgeting for optimal inference cost and latency.
Short-Term Buffer
Active conversation window (16K-128K tokens). Maintains immediate turn-taking context with sliding window eviction and importance scoring.
Vector Context
High-dimensional embeddings stored in NexusDB. Enables RAG-style retrieval, cross-session fact consistency, and dynamic knowledge injection.
Long-Term Memory
Structured summaries of past interactions. Auto-compressed using NLP pipelines to preserve key decisions, preferences, and project milestones.
State & Tools
Persistent execution state, tool call history, and session variables. Maintains workflow continuity across multi-step agent orchestration.
Context Window Management
| Strategy | Trigger | Behavior |
|---|---|---|
| Sliding Window | Token limit reached | Removes oldest low-importance turns while preserving system prompts and key anchors. |
| Semantic Compression | Context > 50% capacity | Summarizes redundant messages into dense vectors or brief textual anchors. |
| Dynamic Injection | Query intent detected | Pulls relevant chunks from vector store and inserts them as temporary context. |
| Explicit Anchors | User-defined | Pinned messages that survive all eviction cycles. Useful for system instructions or critical constraints. |
Data Flow Pipeline
Input Stream
User prompt / Agent action
Intent Router
Classifies context needs
Memory Resolver
Fetches relevant states
Context Assembler
Builds optimized prompt
Inference Engine
Generates response
Component Breakdown
The pipeline operates asynchronously to avoid blocking inference. Each stage applies strict latency budgets (typically < 45ms per hop).
Intent Router
Lightweight classifier that determines whether a query requires historical context, tool execution, or simple pattern matching. Uses a distilled 7B model for sub-10ms routing.
Memory Resolver
Queries NexusDB for semantic matches, checks episodic summaries, and validates tool state. Applies relevance scoring and deduplication before returning candidates.
Context Assembler
Structures the final prompt template. Merges system instructions, retrieved chunks, conversation history, and tool schemas while enforcing token caps and priority rules.
Agent Configuration
Define memory behavior per agent or workspace using YAML or the dashboard. All settings support hot-reloading without service interruption.
# NexusAI Agent Memory Configuration
memory:
context_window: "128k"
eviction_policy: "importance_scored"
short_term:
max_tokens: 65536
compression_threshold: 0.75
anchor_prompts: true
long_term:
enabled: true
storage_backend: "nexusdb-vector-v3"
summary_model: "nexus-summarizer-tiny"
retention_days: 365
auto_index: true
tools_state:
persist_calls: true
max_history: 50
encrypt_pii: trueEnvironment Variables
| Variable | Type | Default | Description |
|---|---|---|---|
NEXUS_MEM_ENABLED | bool | true | Global memory toggle. Disables all persistence if false. |
NEXUS_VECTOR_DIM | int | 1024 | Embedding dimensionality for semantic search. |
NEXUS_CTX_PRIORITY | string | "balanced" | Context weighting strategy: "latency", "accuracy", or "balanced". |
Memory API Endpoints
Programmatic control over context state, retrieval, and eviction. All endpoints return JSON and support standard auth headers.
// Inject custom context chunks into active session
import { NexusClient } from '@nexus/sdk';
const client = new NexusClient({ apiKey: process.env.NEXUS_KEY });
await client.memory.inject({
agentId: "agnt_8x92k",
session: "sess_live_44a",
chunks: [
{ type: "fact", content: "User prefers UTC timestamps", priority: 0.9 },
{ type: "constraint", content: "Do not modify production DB directly", priority: 1.0 }
],
persist: true
});
console.log("Context updated successfully.");// Returns real-time memory utilization
{
"total_tokens": 42150,
"max_capacity": 131072,
"vector_chunks": 1240,
"episodic_summaries": 18,
"evictions_last_hour": 3,
"latency_p95_ms": 24
}Webhooks & Events
Subscribe to memory lifecycle events for observability and custom logic:
memory.context.evicting— Fires before window compressionmemory.vector.indexed— New chunks successfully embeddedmemory.episodic.summarized— Long-term summary generatedmemory.state.persisted— Tool/session state saved to disk
Need advanced context tuning?
Our ML engineers can help you optimize memory allocation, custom compression models, and multi-agent context sharing.
Talk to Engineering →