Software Engineer, Inference (AI)

TL;DR

Software Engineer, Inference (AI): Scaling and optimizing OpenAI’s inference infrastructure across emerging GPU platforms with an accent on advancing inference performance on AMD accelerators. Focus on debugging and optimizing distributed inference workloads across memory, network, and compute layers.

Location: Onsite in San Francisco

Company

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity.

What you will do

• Own bring-up, correctness and performance of the OpenAI inference stack on AMD hardware.
• Integrate internal model-serving infrastructure into a variety of GPU-backed systems.
• Debug and optimize distributed inference workloads.
• Validate correctness, performance, and scalability of model execution on large GPU clusters.
• Collaborate with partner teams to design and optimize high-performance GPU kernels.
• Build, integrate and tune collective communication libraries for model execution.

Requirements

• Experience writing or porting GPU kernels using HIP, CUDA, or Triton.
• Familiarity with communication libraries like NCCL/RCCL.
• Experience with distributed inference systems.
• Ability to solve end-to-end performance challenges.
• Excitement to build new infrastructure from first principles.