building ai infrastructure at scale
dec 2024
shipping mcp gateway in 4 days, building platforms for 3,500+ teams, and why production readiness is about boring decisions.
the challenge
when we started building GenAI infrastructure at Booking.com, the challenge wasn't just technical—it was organizational. how do you give 3,500+ engineering teams access to AI capabilities without creating chaos?
ContextRS: the foundation
i built ContextRS, a Rust-based Model Context Protocol server. why Rust? performance, safety, and the ability to handle concurrent requests from thousands of teams without breaking a sweat.
the key insight: treat AI access like you'd treat any other critical infrastructure. governance first, innovation second.
shipping MCP Gateway in 4 days
during a 4-day accelerator, we shipped MCP Gateway—a production API layer with:
- centralized authentication
- SOX-compliant observability
- usage telemetry
- rate limiting and quotas
how? by keeping it boring. no fancy architectures, no over-engineering. just solid, tested patterns that we knew would work.
lessons learned
1. boring is beautiful - when you're building critical infrastructure, novelty is the enemy
2. governance enables speed - the right guardrails let teams move faster, not slower
3. start with the constraints - SOX compliance, PCI requirements—these aren't afterthoughts
what's next
Agent Foundry is our next bet—enabling teams to deploy production AI agents without building custom infrastructure. but that's a story for another post.