Software Engineer (AI)

TL;DR

Software Engineer (AI): Building core backend services and distributed infrastructure for high-scale generative AI model inference with an accent on system design, performance optimization, and cross-cloud management. Focus on developing customer-facing APIs, orchestration systems, and reliable GPU cluster management to support the next generation of AI infrastructure.

Location: Must be based in New York, NY or San Mateo, CA

Salary: $175,000–$220,000 USD

Company

A Series C, $4B valued startup building cutting-edge generative AI infrastructure for high-speed model inference.

What you will do

• Build and scale backend services for AI model training and inference.
• Design distributed systems from concept to production, ensuring high reliability.
• Optimize cross-cloud infrastructure and GPU cluster management for global scale.
• Develop developer tooling and enterprise features like access control and billing.
• Collaborate directly with customers to identify and solve real-world problems.

Requirements

• 5+ years of experience building backend or infrastructure systems.
• Solid understanding of distributed systems fundamentals.
• Hands-on experience with major cloud platforms (AWS, GCP, or Azure).
• Experience with ML/AI tooling or production-ready platforms.
• Comfort working effectively within both human-led and AI-automated workflows.

Nice to have

• Expertise in large-scale ML infrastructure.
• Proficiency with Kubernetes and cloud-native stacks.
• Experience with IaC (Infrastructure as Code) or CI/CD pipelines.
• Active contributor to open-source software.

Culture & Benefits

• Meaningful equity participation in a rapidly growing, high-impact startup.
• Competitive base salary with comprehensive benefits package.
• Opportunity to work on bleeding-edge technology at the forefront of the AI revolution.
• Collaborative environment with engineers from Meta PyTorch and Google Vertex AI.
• Focus on ownership and autonomy with minimal bureaucracy.