Member of Technical Staff, Inference Performance (AI)

Location: On-site in San Francisco, California, 5 days per week

Salary: $200,000 annual base salary plus equity

Company

AI infrastructure startup building a high-performance inference cloud for open models, processing trillions of tokens per month.

What you will do

• Profile and optimize inference latency, throughput, and memory usage.
• Improve serving engines through batching, caching, quantization, and speculative decoding.
• Develop and tune CUDA, HIP, and Triton kernels.
• Operate heterogeneous accelerator clusters and benchmark models and hardware under production workloads.
• Build observability and reliability into the serving platform.
• Debug performance across models, runtimes, kernels, networking, and hardware.

Requirements

• Demonstrated experience with high-performance AI inference or model-serving systems.
• Experience with GPU kernel development using CUDA, HIP, or Triton.
• Knowledge of quantization, speculative decoding, batching, or KV-cache optimization.
• Experience with PyTorch, vLLM, SGLang, TensorRT-LLM, or similar frameworks.
• Background in GPU, accelerator, HPC, or distributed-compute infrastructure.
• Ability to explain personal optimizations, measurement methods, and production impact.

Culture & Benefits

• Highly hands-on environment with broad technical ownership.
• Direct access to founders.
• Opportunity to work across models, kernels, hardware, and infrastructure.
• Equity included in the compensation package.