Infrastructure Engineer (AI)

TL;DR

Infrastructure Engineer (AI): Building the core infrastructure for AI evaluation systems, including AI gateways, runtimes, and serving layers with an accent on low-latency APIs and high-reliability distributed systems. Focus on designing enterprise-grade infrastructure, implementing deep observability, and scaling AI-centered products.

Location: Hybrid (Bay Area)

Company

hirify.global is a platform for evaluating how AI models perform in the real world, founded by researchers from UC Berkeley's SkyLab.

What you will do

• Design and implement low-latency, high-reliability APIs for leaderboards, models, and hirify.globals.
• Build enterprise-grade infrastructure including rate limiting, authentication, usage metering, and SOC 2 compliance.
• Implement deep observability using distributed tracing, latency breakdowns, and real-time dashboards.
• Integrate core evaluation platform data and customer-specific benchmarks into AI-centered products.
• Collaborate with research teams to transform novel ideas into full-featured products.
• Orchestrate API-based products from the ground up.

Requirements

• 6+ years of backend engineering experience with a focus on distributed systems or infrastructure.
• Strong proficiency in Go and/or Rust for building high-throughput APIs or proxy systems.
• Experience with LLM provider APIs (OpenAI, Anthropic, Google) and handling streaming and rate limits.
• Proficiency with AWS or GCP, Kubernetes, Terraform, Postgres, and Redis.
• Product-oriented mindset regarding developer experience.
• Must be based in or able to work hybrid in the Bay Area.

Nice to have

• Experience building API gateways or proxies (Kong, Envoy, Tyk, etc.).
• Background in ML infrastructure, model serving, or evaluation frameworks.
• Experience building enterprise features like SSO, RBAC, or multi-tenancy.
• Experience with billing infrastructure (Stripe, Metronome, Orb).
• Familiarity with vLLM, LiteLLM, or LangChain.

Culture & Benefits

• Competitive compensation and equity aligned with local markets.
• Comprehensive medical, dental, and vision health benefits.
• Opportunity to work on cutting-edge AI with a small, mission-driven team.
• Culture emphasizing transparency, trust, craftsmanship, and community impact.