Applied AI Engineer (Inference)

TL;DR

Applied AI Engineer (Inference): Building and optimizing high‑performance model serving capabilities for real production workloads with an accent on benchmarking and system profiling. Focus on identifying bottlenecks in the inference stack, driving targeted optimizations for throughput and latency, and evaluating model‑serving techniques.

Location: Must be based in Bellevue, WA, San Francisco, CA, or Sunnyvale, CA. Must be a U.S. person (Citizen, Green Card holder, etc.) due to export control regulations.

Salary: $188,000 – $275,000

Company

CoreWeave is the Essential Cloud for AI, providing specialized infrastructure and tools to help innovators build and scale AI with confidence.

What you will do

• Build and maintain benchmarking workflows to measure latency, throughput, and quality regressions.
• Profile model-serving behavior to find bottlenecks in prefill, decode, KV cache usage, and batching.
• Drive targeted optimization efforts for specific customer and product workloads.
• Design experiments on quantization, speculative decoding, and caching strategies.
• Partner with inference platform engineers to productionize performance improvements.
• Produce technical writeups and recommendations for model configurations and hardware allocation.

Requirements

• 4+ years of experience in machine learning, systems, or performance engineering.
• Strong programming skills in Python and experience in production environments.
• Familiarity with LLM inference stacks such as vLLM, SGLang, or TensorRT-LLM.
• Understanding of tradeoffs between latency, throughput, GPU utilization, and quantization.
• Must be a U.S. person per government export regulations.

Nice to have

• Experience optimizing inference workloads on modern GPU hardware.
• Proficiency with profiling tools like Nsight Systems or PyTorch profilers.
• Experience with benchmark suites for coding, reasoning, or agent workloads.
• Experience using real production traces to guide optimization work.

Culture & Benefits

• 100% company-paid medical, dental, and vision insurance.
• 401(k) with generous employer match and ESPP participation.
• Flexible PTO and paid parental leave.
• Daily catered lunch in office and data center locations.
• Mental wellness benefits through Spring Health and family-forming support.