Sr. Engineering Manager, Inference (AI/ML)

TL;DR

Sr. Engineering Manager, Inference (AI/ML): Leading the group responsible for productizing and operating the inference offering with an accent on service reliability, observability, and developer-facing tooling. Focus on building scalable inference systems, coordinating cross-functional roadmap execution, and improving the overall developer experience for AI workloads.

Location: Remote (US) or hybrid in Livingston, NJ; New York, NY; San Francisco, CA; Sunnyvale, CA; or Bellevue, WA. Candidates must be a U.S. person (citizen, lawful permanent resident, refugee, or asylee) due to export control regulations.

Salary: $188,000 – $303,000

Company

CoreWeave is The Essential Cloud for AI™, providing high-performance infrastructure for AI labs and enterprises to build and scale AI with confidence.

What you will do

• Lead and grow the engineering team for the W&B Inference product, focusing on service reliability and orchestration.
• Drive execution on roadmap initiatives in close partnership with Product to deliver measurable customer impact.
• Own engineering processes for the inference productization layer, including incident response, release management, and observability.
• Partner with infrastructure teams to integrate platform capabilities into a cohesive, user-facing product.
• Guide the design and delivery of application-layer enhancements such as tracing and tool call handling.
• Build a culture of ownership, technical excellence, and continuous improvement within the team.

Requirements

• 7+ years of experience in software engineering.
• 3+ years managing or leading engineering teams responsible for distributed systems or large-scale services.
• Must be a U.S. person (U.S. citizen, national, lawful permanent resident, refugee, or asylee) to comply with export control regulations.
• Proficiency in distributed compute concepts, including observability, autoscaling patterns, and API design.
• Experience leading engineering teams through ambiguous, multi-stakeholder projects.

Nice to have

• Background in high-scale systems, real-time APIs, or cloud infrastructure (especially model-serving).
• Experience in platform domains such as IAM, billing/metering, or deployment tooling.
• Expertise in building production-scale inference systems.

Culture & Benefits

• 100% company-paid medical, dental, and vision insurance.
• 401(k) with a generous employer match.
• Equity awards and ability to participate in the Employee Stock Purchase Program (ESPP).
• Flexible PTO and paid parental leave.
• Catered lunch each day in office and data center locations.
• Comprehensive support including mental wellness and family-forming benefits.