1990 Web Archive
1990 Web Archive is an open-source web preservation platform dedicated to capturing and rendering the early internet era (1990β1999). It provides tools for crawling, archiving, and viewing historical web content with pixel-perfect accuracy to the technologies of that time.
π Table of Contents
βοΈ Installation
Prerequisites
- Node.js β₯ 18.0
- npm β₯ 9.0 or yarn β₯ 1.22
- Docker β₯ 24.0 (optional, for containerized deployment)
- At least 50GB free disk space for full archive
From Source
# Clone the repository git clone https://github.com/1990webarchive/1990-web-archive.git cd 1990-web-archive # Install dependencies npm install # Configure environment variables cp .env.example .env # Edit .env with your configuration # Run database migrations npm run migrate # Start the development server npm run dev
Using Docker
# Pull and run with Docker Compose docker compose up -d # Verify the service is running curl http://localhost:3000/api/health
π Quick Start
import { ArchiveClient } from '@1990web/archive-sdk'; const archive = new ArchiveClient({ apiKey: process.env.ARCHIVE_API_KEY, baseURL: 'https://api.1990web.archive/v2', }); // Search for pages from 1995 const pages = await archive.search({ era: '1995', maxResults: 50, sortBy: 'relevance', }); // Render a preserved page const rendered = await archive.render(pages[0].id); console.log(rendered.html); console.log(rendered.timestamp); // '1995-03-14T08:30:00Z'
β¨ Features
| Feature | Status | Details |
|---|---|---|
| Deep Web Crawling | β Live | Crawls archived URLs with protocol-specific handlers |
| Pixel-Perfect Rendering | β Live | Authentic Netscape-era rendering engine |
| GeoCities Recovery | β Live | 180k+ personal homepages restored |
| Advanced Search | β Live | Era filters, technology tags, visual style search |
| Cryptographic Provenance | π Beta | Blockchain-verified archive timestamps |
| Live Preview Mode | π Beta | Interactive browser simulation |
| API v2 REST/GraphQL | π Beta | Full programmatic access |
| AI-Powered Recovery | π§ WIP | ML-based damaged page reconstruction |
| Collaborative Annotations | π Planned | Community-driven page annotations |
ποΈ Architecture
The system is composed of several core services:
| Service | Language | Description |
|---|---|---|
archiver |
TypeScript | Distributed crawler with era-specific browser emulation |
renderer |
Rust + Wasm | Pixel-perfect rendering of legacy HTML/CSS/JS |
search-engine |
Go | Inverted index with era-aware ranking |
storage |
Rust | Distributed blob storage with deduplication |
api-gateway |
Rust | REST + GraphQL API with rate limiting |
provenance |
Go | Cryptographic signing and verification |
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββ # β Client / SDK β # ββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββ # βΌ # ββββββββββββββββββββββββββββββββββββββββββββββββββββββ # β API Gateway (Rust) β # β ββββββββββββ βββββββββββββ ββββββββββββββββββ β # β β REST β β GraphQL β β WebSocket β β # β ββββββ¬ββββββ βββββββ¬ββββββ βββββββββ¬βββββββββ β # βββββββββΌβββββββββββββββΌβββββββββββββββββΌββββββββββββ # βΌ βΌ βΌ # βββββββββββ ββββββββββββ βββββββββββββ # β Search β β Render β βProvenance β # β Engine β β Engine β β Service β # ββββββ¬βββββ ββββββ¬ββββββ βββββββ¬ββββββ # βΌ βΌ βΌ # βββββββββββββββββββββββββββββββββββββββββββββββ # β Distributed Storage β # β (Deduplicated Blobs + Indexes) β # βββββββββββββββββββββββββββββββββββββββββββββββ # β # ββββββββββββββββββββββββββββββββββββββββββββ # βΌ # βββββββββββββββββββββββββββββββββββββββββββββββ # β Archive Nodes (Crawlers) β # β (Emulating Netscape, IE, Mosaic, etc.) β # βββββββββββββββββββββββββββββββββββββββββββββββ # ββββββββββββββββββββββββββββββββββββββββββββββββββ
π‘ API Reference
Base URL
BASE_URL = https://api.1990web.archive/v2
Endpoints
| Method | Endpoint | Description |
|---|---|---|
GET |
/health |
Health check and service status |
POST |
/search |
Search archived pages with era filters |
GET |
/pages/:id |
Retrieve page metadata and content |
GET |
/pages/:id/render |
Get rendered HTML with original styling |
GET |
/pages/:id/provenance |
Cryptographic proof of archival date |
GET |
/collections/:id |
Get curated collection of pages |
POST |
/collections |
Create a new curated collection |
GET |
/stats/era/:year |
Statistics for a specific year |
GET |
/technologies |
List technologies used across archive |
Search Request Example
{ "query": "web ring guestbook", "filters": { "era": "1990-1999", "technology": ["frameset", "gif"], "siteType": "personal" }, "sortBy": "relevance", "page": 1, "perPage": 20 }
Response Example
{ "ok": true, "total": 42187, "page": 1, "perPage": 20, "results": [ { "id": "gct:geocities:8832:index.html", "url": "http://geocities.com/Area51/Cavern/8832/", "snapshotDate": "1997-06-12T00:00:00Z", "title": "β Cool Web Page β - Under Construction", "technologies": ["frameset", "animated-gif", "marquee", "guestbook"], "provenance": "sha256:a8f5f167f44f4964..." } ] }
π Usage Examples
CLI Usage
# Search for pages from the 1990s 1990archive search "web ring" --era 1990s --limit 10 # View details of a specific archived page 1990archive show "gct:geocities:8832:index.html" # Render a page in a simulated browser 1990archive render "gct:geocities:8832:index.html" --browser "netscape3" # Download all pages from a specific GeoCities area 1990archive download "gct:Area51/*" --output "./archive/area51" # Generate a report on technology adoption 1990archive stats --era 1990s --output "report.pdf"
Programmatic Archive Collection
import { CollectionBuilder } from '@1990web/archive-sdk'; // Build a collection of GeoCities pages from 1996 const builder = new CollectionBuilder({ name: 'Geocities Golden Era', description: 'Best of 1996 GeoCities personal pages', }); builder .filter({ era: '1996', siteType: 'personal' }) .filter({ technologies: ['animated-gif', 'marquee'] }) .sortBy('popularity') .limit(100); // Export as HTML (authentic 90s style!) or JSON await builder.export('geo-golden-era', 'html'); console.log(`Exported ${builder.count} pages`); // β Exported 100 pages
π Project Structure
1990-web-archive/ βββ # Core Services β βββ archiver/ # Distributed crawler β β βββ src/ β β βββ protocols/ # HTTP, FTP, Gopher handlers β β βββ renderers/ # Browser emulators β β β βββ renderer/ # Pixel-perfect rendering engine β β βββ src/ β β βββ wasm/ # WebAssembly modules β β βββ fonts/ # Period-appropriate fonts β β β βββ search-engine/ # Go-based search index β βββ storage/ # Distributed blob storage β βββ api-gateway/ # REST + GraphQL gateway β βββ provenance/ # Cryptographic signing β βββ packages/ # SDK & Libraries β βββ archive-sdk/ # TypeScript SDK β βββ cli/ # Command-line interface β βββ react-components/ # React archive viewer β βββ docs/ # Documentation β βββ getting-started.md β βββ api-reference/ β βββ contributing.md β βββ test-data/ # Test fixtures & samples βββ .github/ # CI/CD workflows βββ docker-compose.yml βββ package.json βββ tsconfig.json βββ .env.example βββ README.md
π€ Contributing
We welcome contributions from everyone! Whether you're a web historian, a browser engine developer, or just someone who loves the aesthetic of the early web β there's a place for you here.
Getting Started
- Fork the repository
- Clone your fork locally
- Create a branch for your feature:
git checkout -b feature/amazing-rendering - Commit your changes:
git commit -m 'feat: add animated GIF support' - Push to the branch:
git push origin feature/amazing-rendering - Open a Pull Request
Areas We Need Help With
- π¨ Browser Emulation β Help add support for Mosaic, Internet Explorer 3, Opera 3.x
- π Documentation β Improve docs, add tutorials, write blog posts
- π§ͺ Testing β Expand test coverage for rendering edge cases
- π Protocol Support β Add handlers for FTP browsing, Gopher, WAIS
- ποΈ Storage Optimization β Improve deduplication and compression algorithms
- π Analytics Dashboard β Build visualizations for web history trends
π Code of Conduct
This project adheres to the Contributor Covenant Code of Conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to conduct@1990web.archive.
π₯ Contributors
This project exists thanks to all the people who contribute.
π Contribution Activity
π License
This project is licensed under the MIT License. See the LICENSE file for details.
The early web was built by everyone β let's keep it that way. This archive belongs to the community.
Made with π by the 1990 Web Archive team. Preserving history, one page at a time.
β¬ Back to top