Engineering Manager (GPU Infrastructure)

TL;DR

Engineering Manager (GPU Infrastructure): Leading and operating superclusters to power frontier AI models with an accent on GPU cluster deployment, optimization, and scaling. Focus on overseeing topology-aware scheduling, hardware fault detection, and collaborating with cloud providers to deploy new GPU architectures.

Location: Must be based in the United States or Canada. Remote-friendly with offices in Toronto, San Francisco, New York City, London, Paris, and Montreal.

Company

Cohere is a security-first enterprise AI company building cutting-edge foundation AI models and end-to-end products for real-world business problems.

What you will do

• Lead and mentor a team of engineers specializing in GPU infrastructure, fostering technical excellence.
• Define and execute the technical roadmap for GPU cluster deployment, optimization, and scaling.
• Oversee the implementation of topology-aware scheduling, hardware fault detection, and performance optimization.
• Partner with AI researchers to translate emerging infrastructure needs into robust technical solutions.
• Establish observability and monitoring frameworks for GPU utilization and reliability.
• Manage vendor relationships and contract negotiations for hardware and cloud services.

Requirements

• Experience managing engineering teams with a focus on technical mentorship and growth.
• Deep expertise in ML/HPC infrastructure: GPU/TPU clusters, distributed training frameworks (JAX, PyTorch, TensorFlow).
• Proven experience with Kubernetes at scale for AI workloads in multi-cloud environments.
• Knowledge of infrastructure monitoring tools (Prometheus, Grafana) and IaC tools (Terraform, ArgoCD).
• Experience with cost optimization and capacity planning for GPU infrastructure.
• Must be based in the United States or Canada.

Culture & Benefits

• 6 weeks of paid vacation (30 working days).
• Full health and dental benefits, including a dedicated budget for mental health.
• Retirement matching via RRSP, 401K, or Pension Scheme.
• 100% Parental Leave top-up for up to 6 months for either parent.
• Annual enrichment benefits for arts, culture, fitness, and workspace improvement.
• Remote-friendly culture with a $500 home office stipend and co-working benefits for those not near an office.