Staff Forward Deployed Engineer (AI)

Location: United States; on-site. Base salary: $265,000–$325,000 in California, New York, and Washington; $240,000–$295,000 in other US states; C$340,000–C$415,000 in Canada, plus equity.

Company

Cohere is an enterprise AI company building foundation models and end-to-end products for real-world business applications.

What you will do

• Embed with customer engineering teams to connect Cohere AI models to complex production environments.
• Design multi-tenant model serving, intelligent routing, multi-cluster architectures, and cross-datacenter failover.
• Optimize private and AI-in-a-box deployments across Kubernetes, SRE constraints, and model performance requirements.
• Benchmark and profile customer LLM workloads, using metrics, dashboards, and distributed logs to diagnose performance degradation.
• Support model deployment on GPUs, TPUs, AMD accelerators, and custom ASICs, and establish production SLOs and observability.
• Translate customer deployment challenges into platform upgrades and core product improvements.

Requirements

• 8+ years of engineering experience running production infrastructure, designing large-scale distributed systems, or deploying enterprise software in complex environments.
• Deep production Kubernetes expertise, including multi-cluster orchestration across hybrid and multi-cloud environments such as GCP, Azure, AWS, and OCI.
• Strong understanding of LLM architecture and inference dynamics, including attention mechanisms, compute-bound prefill, and memory-bound decode phases.
• Hands-on experience with inference engines and optimization stacks such as vLLM, TensorRT-LLM, lmd, or Dynamo.
• Familiarity with distributed serving, prefill/decode disaggregation, distributed KV caching, and multi-GPU workloads.
• Customer-facing technical leadership, including scoping ambiguous problems and guiding executive technical stakeholders.
• Ability to write same-day production code while managing client deployment blockers and politics.
• Strong programming foundations in Python, Go, or C++, with the ability to work across backend data pipelines and infrastructure tuning.

Culture & Benefits

• Full health and dental coverage, mental health budget, RRSP matching, 401(k), and pension scheme.
• 100% parental leave top-up for up to six months for either parent.
• Six weeks of paid vacation and annual enrichment benefits covering arts and culture, fitness, quality time, and workspace improvements.
• Education and learning stipend for conferences, courses, and coaching.
• Weekly lunch stipend of $75/£75 or local equivalent, plus office lunch programs, snacks, and community events.
• $500 home office stipend, coworking benefit, office travel budget, and annual company offsite.

Hiring process

• Applicants may be screened and assessed using AI-enabled tools against the position criteria.
• Accommodation requests can be submitted through the provided accommodations form.