Software Engineer - Training/Inference (C++)

Location: Palo Alto, California, or Seattle, Washington, United States

Base salary: $180,000–$440,000 USD per year

Company

Build AI systems designed to understand the universe and support humanity's pursuit of knowledge.

What you will do

• Architect and implement distributed model-serving infrastructure, including load balancing, auto-scaling, batch scheduling, and global KV caching.
• Optimize inference latency, throughput, concurrency, and tail latency under production workloads.
• Benchmark and accelerate inference engines through GPU kernel optimization, code generation, quantization, and speculative decoding.
• Develop tracing, replay, debugging, CI/CD, endpoint deployment, image publishing, and inference-engine update tools.
• Contribute to scaling test-time compute, RL rollouts, and model-hardware co-design.

Requirements

• Deep low-level systems programming experience with C, C++, or Rust.
• Experience building large-scale, highly concurrent production-serving systems.
• Experience with GPU inference engines such as vLLM, SGLang, Triton, or TensorRT-LLM.
• Strong knowledge of batching, caching, load balancing, parallelism, GPU kernels, and code generation.
• Experience with quantization, speculative decoding, distillation, low-precision numerics, testing, benchmarking, reliability, and CI/CD for inference services.

Culture & Benefits

• Work in a small, highly motivated organization with a flat structure and hands-on engineering culture.
• Contribute directly to AI systems serving Grok at massive scale.
• Equity and comprehensive medical, vision, and dental coverage.
• 401(k) retirement plan, disability and life insurance, discounts, and other benefits.