AI Inference SRE (AI)

Location: Remote (US)

Company

hirify.global provides an AI routing and infrastructure layer for accessing, managing, and optimizing large language models across providers.

What you will do

• Build and operate monitoring for provider and endpoint latency, throughput, errors, uptime, and output correctness.
• Define provider-tier SLOs, improve degraded-endpoint detection, and automate traffic failover with the routing team.
• Own provider incident response, including triage, mitigation, external communication, postmortems, and follow-up actions.
• Create provider scorecards, SLO reporting, continuous canaries, and evaluations for silent inference regressions.
• Automate endpoint operations such as capacity changes, deprecations, rate-limit tuning, and endpoint disabling.
• Develop load-testing and launch-readiness tooling for high-volume provider traffic.

Requirements

• 4+ years of experience in SRE, production engineering, or infrastructure roles operating high-traffic customer-facing systems.
• Strong observability experience with metrics, tracing, logs, SLOs, error budgets, and actionable alerting.
• Software engineering skills with TypeScript and/or Python, and a preference for building operational tools.
• Experience with distributed-systems failure modes including timeouts, retries, backpressure, partial outages, and noisy neighbors.
• Ability to lead incidents calmly and communicate clearly with external providers under pressure.
• Understanding of LLM inference concepts such as streaming, tool calling, prompt caching, and throughput/latency tradeoffs.

Nice to have

• Experience at an inference provider, model lab, GPU cloud, API gateway, or CDN company.
• Experience with TypeScript, Cloudflare Workers, Postgres, ClickHouse, GCP, or Vercel.
• Background in routing, load balancing, traffic management, ML evaluations, or synthetic monitoring.

Culture & Benefits

• Remote role within the US.
• Work on infrastructure handling nearly a billion requests and more than 20 trillion tokens per day across 80+ providers.
• Collaborate with Provider Operations and routing teams to improve reliability for enterprise AI workloads.