Назад
4 месяца назад

Senior Machine Learning Engineer, LLM Inference Optimization (AI)

16 267 - 21 850$
Формат работы
remote (только USA)
Тип работы
fulltime
Грейд
senior
Английский
b2
Страна
US
Вакансия из списка Hirify.GlobalВакансия из Hirify RU Global, списка компаний с восточно-европейскими корнями
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Senior Machine Learning Engineer, LLM Inference Optimization (AI): Optimizing production inference services for frontier LLM and VLM models with an accent on latency, throughput, memory efficiency, GPU utilization, model quality, and cost per token. Focus on deploying and extending inference engines, building compression and benchmarking workflows, and solving performance bottlenecks across model, kernel, runtime, scheduler, gateway, and cluster layers.

Location: Palo Alto, California, United States

Base compensation: $195,200–$262,200 USD per year

Company

Nebius builds a full-stack AI cloud platform for data and model training, inference, and production deployment, with infrastructure spanning compute, storage, networking, and applied AI.

What you will do

  • Own optimization for model families, customer endpoints, and serving backends.
  • Deploy, configure, benchmark, and extend inference engines such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, and NVIDIA Dynamo.
  • Optimize LLM and VLM endpoints for latency, throughput, memory efficiency, GPU utilization, quality, and cost per token.
  • Build model-compression workflows using quantization, quantization-aware training, distillation, low-bit serving, and accuracy recovery.
  • Implement inference acceleration techniques including speculative decoding, KV-cache optimization, prefix caching, chunked prefill, continuous batching, and disaggregated serving.
  • Build reproducible benchmark harnesses and collaborate with kernel and platform engineers to diagnose bottlenecks across the serving stack.

Requirements

  • Strong Python and PyTorch engineering skills.
  • Hands-on experience deploying or optimizing LLM, VLM, or high-throughput transformer inference systems.
  • Practical knowledge of at least one modern inference stack, such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, or KServe.
  • Strong understanding of transformer inference bottlenecks, including KV cache, attention, memory bandwidth, batching, parallelism, and long-context serving.
  • Ability to evaluate latency, throughput, quality, utilization, and cost trade-offs quantitatively.
  • Strong communication and collaboration skills across research, kernel, infrastructure, product, and customer teams.

Nice to have

  • Experience with quantization techniques and formats such as FP8, INT8, INT4, NVFP4, MXFP4, AWQ, GPTQ, or SmoothQuant.
  • Experience with distillation, speculative decoding, EAGLE, Medusa, or multi-token prediction.
  • Experience with agentic workloads, tool calling, structured outputs, streaming APIs, high concurrency, or multi-step orchestration.
  • CUDA or Triton familiarity.
  • Open-source contributions to inference and ML infrastructure projects.

Culture & Benefits

  • 100% company-paid medical, dental, and vision coverage for employees and families.
  • 401(k) plan with up to a 4% company match and immediate vesting.
  • Paid parental leave of up to 20 weeks for primary caregivers and 12 weeks for secondary caregivers.
  • Remote work reimbursement of up to $85 per month for mobile and internet.
  • Company-paid disability and life insurance, plus learning opportunities, flexibility, and ownership.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →