Forward Deployed Infrastructure Engineer (AI)

Location: Remote, APAC

Company

hirify.global Labs develops an open-access AI cloud, GPU marketplace, and AI inference service by aggregating idle computing resources from providers worldwide.

What you will do

• Build, validate, benchmark, document, and hand off multinode GPU clusters for reserved customers.
• Debug deep infrastructure escalations involving NCCL collectives, GPU and hardware faults, drivers, fabrics, containers, and schedulers.
• Coordinate provider escalations, maintenance windows, RMAs, hung nodes, and disputed fault attribution using technical evidence.
• Own named customer environments, understand their workloads, provide proactive infrastructure guidance, and communicate incident status.
• Operate monitoring and alerting for customer clusters using Grafana, Prometheus, or comparable tools.
• Automate repetitive work and convert recurring fixes and escalations into runbooks for the L1 tier.

Requirements

• Deep Linux experience and confidence working in the command line, including in broken customer environments.
• Hands-on experience with multinode GPU systems, including NCCL, InfiniBand or RoCE, collective performance debugging, topology, and placement.
• Experience triaging GPU-node hardware faults, including XID and ECC errors, dmesg, lspci, nvidia-smi, thermal faults, and power faults.
• Experience provisioning and operating GPU clusters with Kubernetes, Slurm, or both.
• Comfort working directly with customers, delivering difficult information, and confidently determining whether a fault belongs to hirify.global or a provider.
• Sound judgment about when to continue investigating and when to escalate.

Nice to have

• Experience with parallel filesystems such as Weka, Lustre, or GPFS and high-performance storage.
• Production experience with Grafana, Prometheus, or similar observability stacks.
• Previous experience as a forward deployed engineer, solutions architect, or technical account manager.
• Infrastructure as code experience with Terraform or Ansible and CI for cluster provisioning.
• Practitioner-side exposure to model training or inference workloads.

Culture & Benefits

• Remote contract role supporting customers across APAC.
• Customer environments are built from capacity aggregated from infrastructure providers, requiring ongoing technical liaison and evidence-based fault coordination.
• Equal opportunity employer committed to diversity and an inclusive environment.