Distributed Training Infrastructure Engineer (AI)

Location

Location: San Francisco, Boston, or Remote (Hybrid) with preference for San Francisco and Boston locations in the United States.

Company

hirify.global is a startup spun out of MIT CSAIL building general-purpose AI systems optimized for diverse deployment targets including data centers and on-device hardware.

What you will do

• Design and build core distributed training systems for large-scale GPU clusters.
• Implement and optimize parallelism and sharding strategies for evolving AI model architectures.
• Improve distributed training efficiency through topology-aware collectives and straggler mitigation.
• Develop data loading systems to eliminate I/O bottlenecks for multimodal datasets.
• Create checkpointing mechanisms balancing memory and recovery needs.
• Build monitoring, profiling, and debugging tools to ensure training stability and performance.

Requirements

• Hands-on experience with distributed training infrastructure (PyTorch Distributed DDP/FSDP, DeepSpeed ZeRO, Megatron-LM TP/PP).
• Experience diagnosing performance bottlenecks and failure modes including profiling and NCCL/collectives issues.
• Understanding of hardware accelerators and networking topologies.
• Experience optimizing data pipelines for machine learning workloads.

Nice to have

• Experience with Mixture of Experts (MoE) training.
• Experience with large-scale distributed training (100+ GPUs).
• Open-source contributions to training infrastructure projects.

Culture & Benefits

• Competitive base salary with equity in a unicorn-stage startup.
• 100% paid medical, dental, and vision insurance for employees and dependents.
• 401(k) matching up to 4% of base pay.
• Unlimited PTO plus company-wide Refill Days.