4 часа назад
Senior Applied ML Engineer (LLM Routing)
Вакансия из Hirify Global, списка международных tech-компанийДля мэтча и отклика нужен Plus
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Senior Applied ML Engineer (LLM Routing): Building and improving the model that selects the best LLM for each request with an accent on training data, evaluation, calibration, cost, quality, and latency. Focus on production-parity benchmarks, online A/B loops, drift monitoring, speed-aware routing, and reliable real-time decisions under changing serving conditions.
Location: Remote-first, with candidates based in Austria, France, Germany, Italy, the Netherlands, Poland, Spain, the United Kingdom, or the European Union. does not provide visa sponsorship or work permits.
Company
develops cloud and Kubernetes technology, including AI model routing and infrastructure for running LLM workloads.
What you will do
- Own the LLM routing model end to end, including features, objectives, training, calibration, and release cadence.
- Build production-parity evaluations, router comparisons, task-family holdouts, and error-bar analysis for every model change.
- Turn production requests, benchmark rollouts, and override feedback into reliable training data with failure categorization.
- Model task success, token cost, cached-input economics, wall-clock speed, and real serving concurrency in the routing objective.
- Run shadow deployments, A/B tests, rollouts, drift monitoring, and retraining while investigating discrepancies between offline and production results.
- Partner with platform engineers on request-level instrumentation, features, labels, and experiments.
Requirements
- 5+ years of experience in applied ML or ML research, including training, evaluating, and shipping real-time decision models.
- Strong Python and SQL skills for processing and investigating millions of request-level records at scale.
- Experience assessing model quality, benchmark validity, and production evaluation results.
- Understanding of TTFT, decode speed, batching, prefix caching, and concurrency in model serving.
- Experience with Python, PyTorch, SQL at scale, ClickHouse, OpenAI- and Anthropic-compatible LLM APIs, benchmark harnesses, and GitLab CI.
Nice to have
- Experience with LLM routing, RouterBench-class evaluations, and cost-quality-latency Pareto fronts.
- Production experience with calibration, abstention, or conformal methods.
- Experience running online A/B loops with drift monitoring on models affecting real financial outcomes.
Culture & Benefits
- Flexible, remote-first work in a global environment.
- Equity options, competitive salary, equipment budget, and extra days off.
- 10% of work time allocated to personal projects or self-improvement.
- Learning budget covering professional development, conferences, and courses.
- Fast-paced workflow, with most feature projects completed in 1 to 4 weeks.
- Annual hackathon, team-building budget, and company events.
Hiring process
- Recruiter screening call and hiring manager interview.
- Technical system-design interview and live coding.
- Executive culture-check interview; a background check may be conducted at the final stage.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →