Software Engineer (AI Training Infrastructure)

Location: Hybrid in San Francisco or New York, United States

Salary: $165K–$330K plus equity

Company

Baseten provides inference infrastructure and developer tooling that help AI companies deploy cutting-edge models into production.

What you will do

• Architect scalable infrastructure for ML training, including scheduling, storage, and networking.
• Translate complex training requirements from research engineers and developers into technical solutions.
• Design a global training scheduler, reinforcement learning systems, and continuous learning pipelines.
• Improve system reliability, performance, and development velocity across the training stack.
• Collaborate with SRE and Capacity teams on advanced training infrastructure.
• Lead technical discussions, mentor junior engineers, and contribute to the infrastructure roadmap.

Requirements

• Bachelor’s degree or higher in Computer Science or a related field.
• 5+ years of professional experience.
• Proficiency in Go and deep production experience with Kubernetes.
• Advanced knowledge of distributed systems and performance tuning.
• Proven experience designing observability systems.
• Experience with ML/AI workloads and MLOps platforms.

Nice to have

• Experience with distributed storage systems, Python, and major or neo-cloud providers.
• Experience with Temporal, Airflow, or similar workload orchestration platforms.
• Familiarity with distributed training frameworks and technologies such as NCCL, PyTorch, Megatron, NemoRL, VeRL, Axolotl, HF Trainer, FSDP, and DeepSpeed.
• Experience developing AI products, tooling, or agents.

Culture & Benefits

• Competitive compensation with meaningful equity.
• U.S.-only medical, dental, and vision insurance coverage for employees and dependents.
• Flexible PTO and a company-wide winter break.
• Paid parental leave and a fertility and family-building stipend.
• U.S.-only company-facilitated 401(k).
• Exposure to ML startups and opportunities for learning and networking.