Distributed Systems Engineer (PyTorch/CUDA)

Location: Hybrid in London, UK, or remote within the UK

Company

hirify.global develops unified general intelligence through large-scale multimodal models focused on vision and interaction with the physical world.

What you will do

• Design, implement, and optimize distributed training systems for multimodal foundation models across thousands of GPUs.
• Research and implement advanced parallelization approaches, including FSDP, tensor parallelism, pipeline parallelism, and expert parallelism.
• Build monitoring, visualization, and debugging tools for large-scale training runs.
• Improve training stability, convergence, and resource utilization across massive GPU clusters.
• Diagnose the training stack, deliver measurable improvements, and systematize reliable large-scale runs during the first 90 days.

Requirements

• Extensive experience with distributed PyTorch training and parallelism for foundation models.
• Deep understanding of GPU clusters, networking, and storage systems.
• Experience solving advanced distributed-training problems at scale, including FSDP and multi-node training.
• Familiarity with NCCL, MPI, and distributed-systems optimization.

Nice to have

• Strong Linux systems administration and scripting skills.
• Experience managing training runs across more than 100 GPUs.
• Experience with containerization, orchestration, and cloud infrastructure.

Culture & Benefits

• Work on multimodal AI systems intended to understand and operate in the physical world.
• Equal opportunity employment environment.