Назад
3 дня назад

Senior Software Engineer II, Inference (AI)

182 000 - 242 000$
Формат работы
onsite
Тип работы
fulltime
Грейд
senior
Английский
b2
Страна
US
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Senior Software Engineer II, Inference (AI): Building and evolving a Kubernetes-native inference platform for low-latency, highly reliable AI services with an accent on micro-batch scheduling, speculative decoding, KV-cache reuse, and strict P99 SLAs. Focus on designing multi-service architectures, improving tail latency and reliability, and leading capacity planning, autoscaling, graceful degradation, and GPU resource optimization.

Location: Sunnyvale, California or Bellevue, Washington, United States

Base salary: $182,000–$242,000 per year, plus discretionary bonus, equity awards, and benefits based on eligibility.

Company

CoreWeave provides cloud infrastructure, tools, and services for building and scaling AI applications.

What you will do

  • Lead design reviews, define team architecture, and decompose multi-service initiatives into milestones.
  • Define and own SLIs/SLOs, drive post-incident actions, and improve reliability across releases.
  • Implement and measure advanced inference optimizations including micro-batch schedulers, speculative decoding, and KV-cache reuse.
  • Strengthen incident readiness through capacity planning, autoscaling, graceful degradation, rollback, and traffic-shift strategies.
  • Mentor IC1/IC2 engineers, review cross-team designs, and raise coding and testing standards.
  • At IC4 level, own an area spanning multiple services and teams, such as request routing, adaptive scheduling, cost-per-token analytics, or GPU resource isolation.

Requirements

  • IC3: approximately 3–5 years of industry experience; IC4: approximately 5–8 years.
  • Experience building distributed systems or cloud services, with strong coding skills in Python or Go.
  • Deep familiarity with networked systems and performance.
  • Production-scale Kubernetes experience, plus CI/CD and observability stacks such as Prometheus, Grafana, and OpenTelemetry.
  • Practical knowledge of inference internals, including batching, caching, mixed precision with BF16/FP8, and streaming token delivery.
  • Track record of improving P95/P99 tail latency and service reliability through metrics-driven work.
  • Bachelor’s or Master’s degree in Computer Science or a related field, or equivalent practical experience.

Nice to have

  • Contributions to inference frameworks such as vLLM, Triton, TensorRT-LLM, Ray Serve, or TorchServe.
  • Experience with CUDA kernels, NCCL/SHARP, RDMA/NUMA, or GPU interconnect topologies.
  • Experience leading multi-team initiatives or partnering with customers on mission-critical launches.
  • C++ experience.

Culture & Benefits

  • Full-time US-based employment with medical, dental, and vision insurance fully paid by CoreWeave, plus life and disability insurance.
  • 401(k) with employer match, ESPP participation, flexible PTO, paid parental leave, and tuition reimbursement.
  • Health, mental wellness, family-forming, childcare, FSA, and HSA benefits.
  • Casual, collaborative, entrepreneurial environment focused on independent thinking, innovative solutions, and rapid growth.
  • Catered lunches at office and data-center locations.
  • Employment is subject to applicable US export-control requirements for access to technology or source code.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →