2. Core Architecture
A modular, event-driven system designed for high-throughput knowledge ingestion, real-time AI enrichment, and globally distributed content delivery.
System Overview
Aevum Encyclopedia's infrastructure is built on a microservices mesh orchestrated through a service mesh (Istio) and container runtime (Kubernetes). The architecture prioritizes data integrity, low-latency retrieval, and horizontal scalability across all tiers.
All stateful components use consensus-based replication (Raft) to guarantee eventual consistency, while read-heavy paths are served through a multi-layer caching strategy (Redis Cluster → CDN Edge → Regional Cache).
Architecture Diagram
The diagram illustrates a request → process → persist flow. All cross-service communication is encrypted (mTLS) and traced via OpenTelemetry for full observability.
Core Modules
API Gateway & Router
Handles authentication, rate limiting, and request routing. Implements GraphQL federation to unify multiple backend services into a single schema.
AI Enrichment Engine
Processes raw contributions through NLP pipelines. Generates embeddings, extracts entities, resolves ambiguities, and suggests cross-links automatically.
Knowledge Graph Service
Manages the semantic network of concepts. Supports SPARQL queries, pathfinding, and dynamic relationship inference across 2.4M+ nodes.
Verification & Audit Layer
Implements cryptographic hashing for content immutability. Tracks every edit, AI suggestion, and editorial override with full provenance.
Data Pipeline
Content ingestion follows an event-driven architecture. Raw submissions are published to a Kafka topic, where multiple consumers process them in parallel:
┌─────────────┐ ┌──────────────┐ ┌──────────────┐
│ Raw Submit │────▶│ Validation │────▶│ AI Parser │
└─────────────┘ └──────────────┘ └──────────────┘
│
┌─────────────┐ ┌──────────────┐ └──────────────┘
│ CDN Push │◀────│ Graph Sync │◀──── Embedding Gen
└─────────────┘ └──────────────┘
- Validation: Schema checks, spam filtering, and language detection
- AI Parser: NER, sentiment analysis, fact extraction
- Embedding Gen: 1536-dim vectors for semantic search
- Graph Sync: Node creation, edge weighting, relationship indexing
- CDN Push: Edge cache invalidation and asset distribution
Performance Specifications
| Metric | Target | Implementation |
|---|---|---|
| Read Latency (p95) | < 45ms | Redis Cluster + Cloudflare Workers |
| Write Throughput | 12K ops/sec | Kafka + Async DB Writers |
| AI Inference (batch) | 85ms/token | vLLM + TensorRT-LLM |
| Graph Query (p99) | < 120ms | Neo4j Cluster + Caching |
| Uptime SLA | 99.99% | Multi-region Active/Active |
Deployment & Scaling
The platform is deployed across three primary regions (US-East, EU-West, APAC-South) using a multi-cluster Kubernetes setup. Database replication uses synchronous writes for primary regions and asynchronous for disaster recovery.
Auto-scaling policies are configured per tier:
- Stateless services: Scale 0–400 pods based on CPU/Memory & custom QPS metrics
- AI workers: GPU-backed nodes scale based on queue depth & inference backlog
- Database: Read replicas scale horizontally; primary sharded by content hash