Product Manager, Inference Platform (AI)

Location: Hybrid in San Francisco or New York, United States

Salary: $200K–$400K plus equity

Company

Baseten provides AI inference infrastructure and developer tooling for companies bringing advanced models into production.

What you will do

• Work with inference engineering and research teams to deliver new performance capabilities to customers.
• Translate advances in batching, caching, quantization, speculative decoding, and serving architectures into product features.
• Define benchmarks and evaluations for latency, throughput, quality, and cost improvements.
• Understand customer workloads and prioritize high-impact optimizations.
• Make performance features easier to configure, evaluate, and adopt in production.
• Partner with engineering on hardware and model tradeoffs, autoscaling, routing, release readiness, compatibility, and regression testing.

Requirements

• Experience building and shipping AI/ML products or infrastructure where performance, scalability, or production behavior mattered.
• Technical background in software engineering, ML engineering, research, or equivalent hands-on experience.
• Strong understanding of model serving architectures, GPU economics, and latency, throughput, quality, and cost tradeoffs.
• Experience working with researchers or infrastructure engineers to bring technically complex capabilities into production.
• Rigorous approach to benchmarking, experimentation, and performance measurement.
• Strong product judgment and ability to translate deep technical work into clear customer value.

Nice to have

• Hands-on experience with PyTorch, vLLM, TensorRT-LLM, SGLang, NVIDIA Dynamo, or similar technologies.
• Experience with open-source inference projects, model optimization, or ML systems research.

Culture & Benefits

• Competitive compensation with meaningful equity.
• For U.S. employees, 100% medical, dental, and vision insurance coverage for employees and dependents.
• Flexible PTO and company-wide Winter Break from Christmas Eve through New Year's Day.
• Paid parental leave and a fertility and family-building stipend through Carrot.
• For U.S. employees, company-facilitated 401(k).
• Exposure to ML startups and opportunities for learning and networking.