Staff Software Engineer (AI)

TL;DR

Staff Software Engineer (AI): Designing and implementing novel model serving architecture for generative media models with an accent on maximizing throughput and minimizing latency. Focus on building performance monitoring tools, optimizing custom GEMM kernels, and advancing multi-dimensional model parallelism.

Location: Must be based in or be able to relocate to San Francisco

Compensation: $180,000–$250,000 + equity + benefits

Company

An AI infrastructure startup focused on frontier performance for generative media models.

What you will do

• Design and implement novel approaches for model serving architecture using an in-house inference engine.
• Maximize throughput and minimize latency while optimizing resource usage.
• Develop performance monitoring and profiling tools to identify system bottlenecks.
• Collaborate with the Applied ML team and frontier lab customers to improve workload efficiency.
• Optimize hardware performance by working deep in the stack, including custom GEMM kernel development.

Requirements

• Strong foundation in systems programming with expertise in bottleneck identification.
• Deep understanding of the ML infrastructure stack, including PyTorch, TensorRT, and TransformerEngine.
• Knowledge of model compilation, quantization, and serving architectures.
• Fundamental understanding of Nvidia hardware systems.
• Experience with or willingness to learn Triton and lower-level accelerator programming.
• Familiarity with internals of Ring Attention, FA3, and FusedMLP.

Culture & Benefits

• Competitive salary and equity packages.
• Visa sponsorship and relocation assistance to San Francisco provided.
• Comprehensive health, dental, and vision insurance.
• Regular team events and offsites.
• Opportunities for professional growth and learning in a challenging environment.