Staff + Senior Software Engineer, Inference (AI)

Location: Ontario, Canada. Hybrid policy: staff are expected to work from one of the offices at least 25% of the time; some roles may require more office attendance.

Company

Anthropic builds reliable, interpretable, and steerable AI systems designed to be safe and beneficial for users and society.

What you will do

• Design, build, and maintain distributed systems serving Claude to millions of users worldwide.
• Develop intelligent request routing, load balancing, traffic management, autoscaling, and fleet orchestration across thousands of accelerators.
• Build and operate production-grade deployment pipelines for releasing new models reliably.
• Provide high-performance inference infrastructure for machine learning research and next-generation models.
• Integrate new AI accelerator platforms and support inference for new model architectures.
• Analyze observability data and tune performance across multi-region deployments and geographic routing systems.

Requirements

• Significant software engineering experience, particularly with distributed systems.
• Experience with high-performance, large-scale distributed systems and/or machine learning systems at scale.
• Experience with load balancing, request routing, or traffic management systems.
• Familiarity with LLM inference optimization, batching, and caching strategies.
• Proficiency in Python or Rust; familiarity with Kubernetes and AWS, GCP, or Azure is preferred.
• Bachelor’s degree or equivalent education, training, or professional experience in a relevant field.

Culture & Benefits

• Collaborative work across research, engineering, policy, and business teams.
• Competitive compensation and benefits.
• Optional equity donation matching.
• Generous vacation and parental leave.
• Flexible working hours and office collaboration.
• Visa sponsorship is available, with reasonable efforts and immigration-lawyer support for successful candidates.

Hiring process

• Applications are reviewed on a rolling basis with no stated deadline.
• Candidates should follow Anthropic’s guidance on AI usage during the application process.