Member Of Technical Staff - RL Inference (AI)

TL;DR

Member Of Technical Staff - RL Inference (AI): Building and optimizing the inference stack for RL workloads with an accent on performance bottlenecks and low-precision training. Focus on implementing novel RL techniques, optimizing large-scale distributed systems, and enhancing LLM inference efficiency.

Location: Palo Alto, CA

Company

xAI is dedicated to creating AI systems that accurately understand the universe and aid humanity in its pursuit of knowledge.

What you will do

• Design and optimize the inference stack for various RL workloads, from small-scale ablations to production training runs.
• Analyze and profile large-scale RL systems to address performance bottlenecks.
• Collaborate with the modeling team to efficiently implement novel RL algorithms and techniques.

Requirements

• Experience building, debugging, and optimizing large-scale distributed systems.
• Proven experience in LLM inference.
• Proficiency in Python, C++, and/or Rust.
• Experience with frameworks such as PyTorch, Jax, and CUDA.
• Strong communication skills for concise and accurate knowledge sharing.

Nice to have

• Deep knowledge of quantization and numerics in LLM inference and training.
• Experience developing inference engines like SGLang or vLLM.

Culture & Benefits

• Flat organizational structure with a focus on engineering excellence.
• Hands-on environment where leadership is earned through initiative and delivery.
• Culture that thrives on curiosity and challenging oneself.
• Emphasis on strong work ethic and prioritization skills.