Назад
Company hidden
1 день назад

Sr. Inference Engineer (LLM Serving)

Формат работы
hybrid
Тип работы
fulltime
Грейд
senior
Английский
b2
Страна
CR
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Sr. Inference Engineer (LLM Serving) (LLM inference, GPUs): Operating open-weight LLMs and task models on proprietary GPU infrastructure for production AI answers with an accent on serving performance, model evaluation, and cost efficiency. Focus on optimizing batching, caching, quantization, routing, and fine-tuning while maintaining measurable latency, throughput, and answer quality.

Location: Prague, Czech Republic; hybrid work model

Company

hirify.global builds an AI-native data intelligence platform that turns governed business data into answers, reports, agents, and embedded analytics for enterprise customers.

What you will do

  • Deploy, operate, and upgrade open-weight LLMs and task models, including guardrails, PII detectors, classifiers, and judges, on company GPUs across production regions.
  • Optimize inference performance and cost through batching, KV and prefix caching, quantization, and parallelism; measure time to first token, throughput, p95 latency, and cost per answer.
  • Evaluate new open-weight models on real analytics workloads and recommend production candidates.
  • Develop the inference gateway with intent- and complexity-based routing, semantic caching, and validation that escalates weak answers.
  • Adapt models with LoRA adapters and fine-tuning for semantic-layer query generation, root-cause analysis, and narrative reports.

Requirements

  • 5+ years of software engineering experience, including 2+ years running ML or LLM models in production.
  • Hands-on production experience with at least one modern LLM serving engine: vLLM, SGLang, or TensorRT-LLM.
  • Working understanding of inference bottlenecks, including KV cache, attention, memory bandwidth, batching, and parallelism, with the ability to reason about latency, throughput, and cost quantitatively.
  • Experience with Docker, Kubernetes, and GPU instances in a public cloud.
  • Disciplined approach to benchmarking and evaluation; models must be measured before production release.
  • Clear English communication required when working with engineers and product stakeholders.

Nice to have

  • Speculative decoding, multi-LoRA serving, or PEFT fine-tuning.
  • Contributions to vLLM, SGLang, PyTorch, or similar open-source projects.
  • CUDA or Triton kernels and experience with the NVIDIA GPU stack, including drivers and NCCL.
  • LLM routing, guardrail models, embeddings, and rerankers.
  • Go or Rust, Terraform, and Helm.
  • Observability for LLM systems with OpenTelemetry, Langfuse, Prometheus, or Grafana.
  • Experience with analytics, semantic layers, or text-to-SQL.

Culture & Benefits

  • Hybrid work model with flexible working hours.
  • 25 vacation days and personal days.
  • Annual bonus, stock options, meal contribution, mobile phone and tariff, and Edenred Cafeteria points.
  • Company events, GoodLife activities, dog-friendly offices, office snacks and coffee, and bike storage.
  • Laptop for personal use and free tickets to the zoo.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →