Senior Machine Learning Engineer (Inference)

Location: Mountain View, California, USA; in-person work in the Mountain View office with relocation assistance.

Base salary: $250,000–$300,000 annually, plus bonus, equity, and benefits.

Company

Research lab and inference provider focused on realtime AI, first-party speech models, LLM serving, and high-volume modular APIs for consumer applications.

What you will do

• Take models from research through containerization, serving optimization, and reliable production deployment.
• Optimize realtime multimodal inference for latency, throughput, and performance on NVIDIA GPUs.
• Build and scale multi-GPU and multi-node inference systems handling thousands of concurrent connections.
• Profile systems, benchmark unclear problems, and improve architectures, load balancing, and serving reliability.
• Collaborate across model research, infrastructure, and API development to ship stable AI capabilities.

Requirements

• Enough relevant experience to demonstrate strong practical work in backend or machine learning systems, including a PhD in Computer Science, Physics, or Mathematics, or equivalent practical experience.
• Ability to work through ambiguous problems, design benchmarks or prototypes, and take ownership through stable production delivery.
• Strong interest in understanding the fundamental causes of latency, throughput, performance, and reliability issues.

Nice to have

• Deep understanding of modern serving frameworks and inference optimization techniques such as vLLM or TRT-LLM.
• Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding.
• Proficiency in C++, CUDA, Rust, or highly optimized Python, including profiling and optimizing NVIDIA GPU workloads.
• Experience with Kubernetes, Ray, custom load balancing, multi-GPU and multi-node inference, and high-concurrency systems.
• Non-trivial systems programming projects, major inference-engine open-source contributions, or deep-dive technical writing.

Culture & Benefits

• In-person collaboration in the Mountain View office.
• Relocation assistance.
• Flat structure, fast iterations, and minimal process overhead.
• Performance, latency, and reliability are treated as first-class product features.
• Support for sharing work and open-source contributions that advance the field.
• Bonus, equity, and benefits in addition to the base salary.