Назад
3 дня назад

Research Lead, Evaluations (AI)

255 000 - 295 000€
Формат работы
hybrid
Тип работы
fulltime
Грейд
lead
Английский
b2
Страна
Germany
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Research Lead, Evaluations (AI): Building and managing evaluation roadmaps and suites for frontier AI systems with an accent on automated, AI-based, and human evaluations, statistical rigor, and reproducible measurement. Focus on designing benchmarks and objective functions, identifying evaluation blind spots, monitoring training runs, and growing an evaluation research team.

Location: Freiburg, Germany; office attendance required at least 2 days per week or one full week every other week

Compensation: EUR255K–295K + equity, depending on profile

Company

Black Forest Labs is a frontier research lab building multimodal visual intelligence models across images, video, audio, and action, including the FLUX model family.

What you will do

  • Define, maintain, and extend the evaluation roadmap with research and product leads.
  • Build evaluation suites, benchmarks, and objective functions using automated, AI-based, and human evaluation methods.
  • Improve evaluation-pipeline accuracy, reproducibility, trustworthiness, and understanding of error bars.
  • Identify evaluation limitations and blind spots, and communicate findings to project teams and research leadership.
  • Monitor critical training runs independently to detect regressions and emerging capabilities early.
  • Manage and grow the evaluations team, including defining the hiring roadmap and supporting long-term development.

Requirements

  • Proven experience designing, running, and maintaining large, complex evaluation suites for frontier AI systems.
  • Experience with human and automated evaluation approaches, including AI-based methods.
  • Research experience designing and implementing evaluations for AI research.
  • Strong understanding of Python and agentic coding.
  • At least one year of active people-management experience, with an understanding of team leadership beyond an IC role.
  • Strong knowledge of variance, confidence intervals, and significance testing, with sound judgment when interpreting numerical results.
  • Excellent communication skills for explaining evaluation results and their limitations to project leads and research leadership.
  • First-principles thinking and the ability to derive evaluations for new capabilities.

Culture & Benefits

  • Hybrid work from Freiburg with regular use of a real office.
  • Values include scientific rigor, low ego, ambition, shared credit, and kindness.
  • Compensation includes EUR255K–295K plus equity, depending on profile.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →