Infrastructure Engineer (AI)

TL;DR

Infrastructure Engineer (Go/Rust): Building core infrastructure for AI model evaluation systems including AI gateways, automated runtimes, and serving layers with an accent on low-latency APIs and high-reliability distributed systems. Focus on solving streaming challenges, implementing enterprise-grade infrastructure, and scaling AI-centered products.

Location: Hybrid (Bay Area, USA)

Company

hirify.global is a platform for evaluating real-world AI model performance, founded by UC Berkeley researchers to advance the frontier of AI.

What you will do

• Design and implement low-latency, high-reliability APIs for leaderboards, models, and arenas.
• Solve complex streaming problems using SSE across heterogeneous providers, including failure recovery and normalization.
• Build enterprise-grade infrastructure including rate limiting, authentication, usage metering, and SOC 2 compliance.
• Implement deep observability with distributed tracing, latency breakdowns, and real-time dashboards.
• Collaborate with research teams to integrate core evaluation data into full-featured AI products.
• Contribute to the backend of Leaderboards and Evals platforms to unify data architectures.

Requirements

• 4+ years of backend engineering experience in distributed systems, infrastructure, or developer platforms.
• Strong proficiency in Go and/or Rust for building high-throughput APIs or proxy systems.
• Experience with LLM provider APIs (OpenAI, Anthropic, Google) and handling streaming/token management.
• Solid cloud skills with AWS or GCP, Kubernetes, Terraform, Postgres, and Redis.
• Product-oriented mindset and ability to operate comfortably in a fluid startup environment.
• Must be based in the Bay Area for a hybrid work arrangement.

Nice to have

• Experience building API gateways or proxies (Envoy, Kong, Tyk, etc.).
• Background in ML infrastructure, model serving, or evaluation frameworks.
• Experience with enterprise features like SSO, RBAC, and multi-tenancy.
• Knowledge of billing infrastructure (Stripe, Metronome, Orb).
• Familiarity with vLLM, LiteLLM, or LangChain.

Culture & Benefits

• Competitive compensation and equity aligned to market rates.
• Comprehensive health and wellness benefits (medical, dental, vision).
• Opportunity to work on cutting-edge AI within a small, mission-driven team.
• Culture valuing transparency, trust, and community impact.