Engineering Manager, Inference Infrastructure (AI)

Location: San Francisco, CA; New York City, NY; or Seattle, WA. Hybrid work is required, with staff expected to be in an office at least 25% of the time.

Annual salary: $405,000–$625,000 USD

Company

Anthropic develops reliable, interpretable, and steerable AI systems with a focus on safety and social benefit.

What you will do

• Own the technical roadmap for the inference fleet control plane, including traffic routing, capacity placement, cache placement, demand response, and synchronization with inference engines.
• Lead ML platform, infrastructure, and distributed-systems engineers responsible for the request-to-model path.
• Partner with product, inference, performance, and capacity teams to deliver measurable throughput, latency, utilization, and cost improvements.
• Set technical strategy across heterogeneous hardware, multiple cloud providers, and all serving surfaces.
• Run operational practices including on-call rotations, incident response, postmortems, capacity events, and deployment safety.
• Build and retain strong teams, hire to a high technical bar, coach engineers, and shape team structure as the organization grows.

Requirements

• Engineering management experience leading critical-path production infrastructure teams at scale.
• Deep systems experience with areas such as load balancing, scheduling, cluster orchestration, autoscaling, distributed state, high-performance networking, or fleet control planes.
• Experience shipping and quantifying performance or efficiency improvements, including cost impact.
• Experience operating production infrastructure with on-call, incident response, capacity events, and disciplined deployments.
• Ability to make architectural decisions, collaborate across team boundaries, and balance throughput, latency, cost, stability, and delivery speed.
• Bachelor’s degree or equivalent education, training, or experience in a relevant field.

Nice to have

• 5+ years of engineering management experience.
• Experience with LLM inference serving, including KV caching, continuous batching, request scheduling, or prefill/decode disaggregation.
• Experience with Kubernetes internals, cluster schedulers, autoscalers, load balancers, service meshes, or Borg-style systems.
• Experience operating workloads across multiple clouds, heterogeneous accelerator fleets, supercomputing environments, or hyperscaler infrastructure.
• Experience leading multiple teams through rapid growth, hiring, onboarding, and team structuring.

Culture & Benefits

• Collaborative work across research, engineering, policy, and business disciplines.
• Flexible working hours and a hybrid office environment.
• Competitive compensation with optional equity donation matching.
• Generous vacation and parental leave.
• Visa sponsorship is available, with immigration-lawyer support, subject to role and candidate eligibility.