Назад
Company hidden
3 дня назад

Applied AI Engineer (AI)

Формат работы
hybrid
Тип работы
fulltime
Грейд
senior
Английский
b2
Страна
Poland
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Applied AI Engineer (AI): Building and improving evaluation systems for Handle's AI agent with an accent on LLM evals, agent behavior, and production quality. Focus on designing benchmarks and quality gates, running controlled experiments, diagnosing regressions, and restructuring skills or agents when local fixes are insufficient.

Location: Hybrid in Warsaw, Poland

Company

hirify.global connects startups and disruptive technology companies with local engineering talent and supports R&D work through employment, HR, and benefits services.

What you will do

  • Build and own Handle’s evaluation suite, including scenarios, rubrics, graders, benchmarks, and quality gates.
  • Form research hypotheses and run controlled experiments to separate model effects from prompt, tool, skill, and agent-structure effects.
  • Version skills and changes, rerun evaluations, diagnose regressions, and verify improvements.
  • Evaluate conversation quality, style, user understanding, and task reliability.
  • Test new models, assessing portability, failure modes, and when a simpler single-model strategy is preferable.
  • Recommend agent or skill restructuring when repeated failures cannot be solved through local fixes.

Requirements

  • Proven experience building and running evaluations for an LLM product in production, including the ability to write code.
  • Deep interest in agent behavior, model capabilities, and evaluation.
  • Technical fluency to run experiments, inspect failures, and collaborate closely with engineers.
  • Ability to move between theory and concrete product problems.
  • Proactive approach: notice patterns, propose better approaches, and follow them through.

Nice to have

  • Hands-on experience with post-training or fine-tuning open-weight models.

Culture & Benefits

  • Supportive, well-organized, and fair working environment.
  • Benefits package provided through hirify.global; details are shared during interviews.
  • B2B engagement arrangement.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →