ML Platform Engineer, Distributed Training

Location: Remote

Company

hirify.global delivers engineering and AI/ML solutions, with this role embedded with a healthcare and life-sciences AI client.

What you will do

• Work with the client infrastructure team on a GKE training platform covering GPU node pools, job queueing, scheduling, quota, and capacity from the training workload perspective.
• Diagnose platform failures at root cause and tune multi-node GPU networking, including RDMA, GPUDirect, and NCCL, for training throughput.
• Run, scale, and debug multi-node distributed training with PyTorch, FSDP, DeepSpeed, Megatron or NeMo-style approaches, and Ray Train.
• Profile workloads and remove bottlenecks across compute, memory, input pipelines, and interconnects.
• Design performance benchmarks and model evaluation harnesses covering throughput, scaling efficiency, cost per run, and platform or training-recipe changes.
• Instrument training and platform systems with telemetry and monitoring, maintain end-to-end pipeline tests, own the area, and communicate design decisions, status, and risks to engineers and scientists.

Requirements

• 5+ years of engineering experience with recent hands-on work on GPU training infrastructure and large-scale training.
• Production-quality Python software engineering, including tests and code review; coding is central to the role.
• Hands-on Kubernetes on Google Cloud with GKE GPU node pools, job queueing such as Kueue, and capacity and quota management.
• Personal experience running and debugging multi-node distributed training jobs with PyTorch and Ray Train.
• Hands-on knowledge of GPUDirect/RDMA, NCCL tuning, and multi-node GB200 or H200 topology.
• Experience designing performance benchmarks and model evaluations and using their results to make decisions.
• Deep understanding of system operation and design, with the ability to explain how and why systems work without relying on AI coding tools for reasoning.
• A strong software engineering foundation, including clean maintainable code and understanding of the full SDLC.
• A visible track record of ownership, communication, accountability, and effectiveness during changing priorities and team structures.
• Mastery of shell scripting; extensive experience with Google Gemini, GPT-4, or LLaMA; deep knowledge of prompt engineering and fine-tuning; expertise with vector databases and semantic search; hands-on Google Cloud Vertex AI experience; scalable software architecture experience; and proficiency with LangChain, LlamaIndex, or similar orchestration layers.

Nice to have

• Healthcare or life-sciences data experience, including genomics, clinical, or biomedical data, and experience working under HIPAA.
• Observability and telemetry experience for training platforms, including metrics, tracing, alerting, and end-to-end pipeline testing.

Culture & Benefits

• Remote workplace.
• Collaboration with hirify.global engineers, client infrastructure teams, and client scientists.
• Full-time or contract engagement options.
• Compensation varies according to experience, expertise, interview performance, geographic location, and final leveling.

Hiring process

• The role may be hired at multiple levels.
• Final leveling is determined during interviews based on experience, skills, and interview outcomes.
• Standard hiring verification may include background checks, employment verification, and other relevant checks.