Machine Learning Engineer (AI Inference)

Location: San Francisco Bay Area, Mountain View, CA, Greater Boston, or San Jose, CA

Salary: $275,000–$300,000 annually

Company

Stanford-spun AI company building infrastructure for a production AI platform.

What you will do

• Architect high-performance infrastructure for large language model inference.
• Optimize latency, throughput, and cost across production workloads.
• Scale inference across multi-GPU and multi-node environments using Kubernetes.
• Improve GPU scheduling, allocation, and utilization.
• Profile compute, memory, and networking bottlenecks.
• Optimize batching, quantization, KV caching, and parallelism.

Requirements

• Background in ML systems, distributed computing, high-performance computing, or performance engineering.
• Strong Python and/or C++ skills.
• Experience with PyTorch and GPU computing.
• Ability to reason about GPU utilization, memory bandwidth, communication overhead, and model architecture.
• Experience with CUDA, NCCL, Triton, vLLM, TensorRT-LLM, or SGLang is relevant.

Culture & Benefits

• Deeply technical work focused on production AI infrastructure and hardware-level performance.
• Work with GPUs, distributed systems, and large-scale LLM inference.