Research Infrastructure Engineer (AI)

Location: San Francisco, United States; on-site

Company

Windsurf builds Devin, an AI software engineer, and develops systems for advanced AI research and deployment.

What you will do

• Build and operate distributed training infrastructure for large-scale GPU clusters, including job launchers, checkpointing, recovery, fault tolerance, and monitoring.
• Scale coding-agent rollouts across VM sandboxes and support reinforcement learning workloads with hundreds of thousands of concurrent executions.
• Profile and optimize training throughput, data loading, communication, memory utilization, and compute efficiency.
• Design experiment orchestration, tracking, analysis, and research tooling that reduces iteration time.
• Build reliable, high-throughput data pipelines for training and evaluation with strong reproducibility and data quality.
• Diagnose failures across GPUs, networking, numerics, and data while implementing parallelism strategies and graceful recovery.

Requirements

• Deep experience building and operating distributed training systems for large models, from cluster infrastructure through the training loop.
• Strong systems engineering fundamentals across distributed systems, networking, storage, and the hardware-software stack.
• Proficiency in Python and C++, with systems-level experience using PyTorch or equivalent deep learning frameworks.
• Hands-on experience with GPU profiling, memory optimization, compute efficiency, and data, tensor, pipeline, or sequence parallelism.
• Strong debugging skills for complex, non-deterministic distributed systems and sufficient machine learning knowledge to work directly with researchers.
• Demonstrated capability is valued more than credentials; a PhD is only one possible signal.

Culture & Benefits

• Small, highly selective team where research and product development move together.
• Prototypes can reach real deployment quickly.
• Infrastructure operates across thousands of GPUs with access to the systems required for the work.
• Environment emphasizes speed, autonomy, technical depth, and minimal process overhead.