Machine Learning Engineer (LLM Inference Serving, vLLM)

Location: Santa Clara, California, United States — onsite

Salary: $150,000–$190,000 per year

Company

Stealth AI infrastructure startup developing systems for large language model inference and serving.

What you will do

• Build and optimize LLM inference serving systems.
• Refactor inference framework internals to improve performance.
• Manage the KV cache lifecycle.
• Optimize distributed serving throughput and latency.
• Work on scheduling and inference orchestration using Ray, Dynamo, or similar platforms.

Requirements

• Deep hands-on experience with vLLM or SGLang.
• Public GitHub contributions to vLLM or SGLang.
• Experience improving inference framework performance.
• Experience with KV cache lifecycle management.
• Experience with Ray, Dynamo, or similar inference orchestration platforms.
• Ability to work onsite in Santa Clara, California.

Culture & Benefits

• Work on AI infrastructure for LLM inference.
• Focus on distributed serving, scheduling, throughput, and latency optimization.