Product Manager (AI Infrastructure)

Location: Hybrid in San Francisco or New York, United States

Salary: $235,000–$335,000 annually, plus equity.

Company

Baseten provides production inference infrastructure and developer tooling for AI companies, combining applied AI research, flexible infrastructure, and systems for deploying models at scale.

What you will do

• Own workload scaling and placement policies, including autoscaling, regional preferences, compliance requirements, and capacity management.
• Build reliable production inference through healthy-replica routing, rolling deployments, failover, and recovery from unhealthy replicas.
• Define multi-region routing and active-active failover policies.
• Develop the release engine for canary, shadow, and A/B deployments, including traffic shifting, warm-ups, draining, and probes.
• Improve latency, throughput, uptime, cost efficiency, and mean time to recovery for AI serving infrastructure.
• Set the infrastructure product roadmap and own capabilities from backend behavior through customer configuration and observability.

Requirements

• 8+ years of product management experience, including deep experience with infrastructure, distributed systems, or ML serving.
• Strong understanding of scaling, routing, failover, and cost-performance trade-offs at a level suitable for working with staff engineers.
• Experience owning products end to end, from backend behavior through user experience.
• Experience driving cross-team roadmaps and managing dependencies.
• Comfort defining a product category that is still emerging.

Nice to have

• Hands-on experience with GPU infrastructure.
• Experience with Kubernetes.
• Experience with serving frameworks such as vLLM, TensorRT-LLM, or SGLang.

Culture & Benefits

• Competitive compensation with meaningful equity.
• U.S.-only: 100% medical, dental, and vision insurance coverage for employees and dependents.
• Flexible PTO and a company-wide Winter Break from Christmas Eve through New Year's Day.
• Paid parental leave and a fertility and family-building stipend.
• U.S.-only: Company-facilitated 401(k).
• Exposure to ML startups and opportunities for technical learning and networking.