Staff+ Software Engineer (AI)

TL;DR

Staff+ Software Engineer (AI): Leading the technical direction for the shared, accelerator-agnostic inference runtime stack with an accent on performance, correctness, and system abstractions. Focus on architecting scalable infrastructure for heterogeneous hardware (GPUs, TPUs, Trainium) and driving technical alignment across inference and infrastructure teams.

Location: Must be based in or able to commute to San Francisco, CA, Seattle, WA, or New York City, NY (Hybrid: 25%+ in-office required)

Salary: $405,000 - $485,000 USD

Company

Anthropic is an AI safety and research company dedicated to building reliable, interpretable, and steerable AI systems.

What you will do

• Set technical direction and own the architecture for the shared inference runtime stack.
• Develop and evolve accelerator-agnostic interfaces and internal boundaries using Rust and Python.
• Optimize accelerator utilization, scheduling, and memory management across GPU, TPU, and Trainium platforms.
• Build robust validation surfaces including partitioned builds, change-scoped testing, and canary/shadow deployment mechanisms.
• Collaborate with infrastructure teams on compilers, build systems, and toolchains to meet performance requirements.
• Mentor engineers through design reviews, code reviews, and cross-functional collaboration.

Requirements

• Must be based in or able to commute to San Francisco, Seattle, or New York City.
• Deep background in systems engineering or ML infrastructure with hands-on performance profiling and debugging experience.
• Significant experience in high-performance, large-scale distributed systems.
• Proven depth in at least one accelerator ecosystem (CUDA, TPU, or AWS Neuron).
• Track record of defining engineering metrics and SLOs to drive platform improvements.
• Strong communication skills with the ability to influence technical direction across organizational boundaries.

Nice to have

• 8+ years of software engineering experience with significant time as a technical lead.
• Experience with ML compiler toolchains like XLA, Triton, or NeuronX.
• Background in operating production validation surfaces at scale (shadow traffic, canary populations).
• Familiarity with Kubernetes-based development and job scheduling.

Culture & Benefits

• Competitive compensation and benefits package.
• Flexible working hours and generous vacation/parental leave.
• Collaborative research-driven environment with frequent technical discussions.
• Optional equity donation matching.
• Support for visa sponsorship for qualified candidates.