Putting Wikipedia on turbopuffer

maxhalford25@gmail.com (Max Halford) — Wed, 04 Nov 2026 00:00:00 +0000

TLDR: People edit Wikipedia pages all the time. We want a search index that keeps in sync. We do that with Redpanda Connect (ingestion) + turbopuffer (indexing) + a homemade frontend. We manage a 20s p99 ingestion delay, costing ~50$/month.

Today is “h4x day” at turbopuffer. It’s basically a 24-hour hackathon fueled by good food, electrolytes, and high beats per second music. Everyone picks a project, ideally something they would not have tackled in their regular work stream. It’s very energizing. I’m grateful to be surrounded by so many smart and thoughtful people. I feel clumsy around my new colleagues.

Nlp on Max Halford

Putting Wikipedia on turbopuffer