Чётко определённая руководящая роль в быстрорастущей лаборатории frontier AI: здесь можно влиять на исследования, отвечая и за техническое направление, и за развитие команды.
?Регулярный офис
Передовой AI
Понятная зона ответственности
Современный стек
Развитие команды
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Подключить Plus
Описание вакансии
Текст:
Hirify AI/Оригинал
TL;DR
Research Lead, Evaluations (AI): Building and managing evaluation roadmaps and suites for frontier AI systems with an accent on automated, AI-based, and human evaluations, statistical rigor, and reproducible measurement. Focus on designing benchmarks and objective functions, identifying evaluation blind spots, monitoring training runs, and growing an evaluation research team.
Location: Freiburg, Germany; office attendance required at least 2 days per week or one full week every other week
Compensation: EUR255K–295K + equity, depending on profile
Company
Black Forest Labs is a frontier research lab building multimodal visual intelligence models across images, video, audio, and action, including the FLUX model family.
What you will do
Define, maintain, and extend the evaluation roadmap with research and product leads.
Build evaluation suites, benchmarks, and objective functions using automated, AI-based, and human evaluation methods.
Improve evaluation-pipeline accuracy, reproducibility, trustworthiness, and understanding of error bars.
Identify evaluation limitations and blind spots, and communicate findings to project teams and research leadership.
Monitor critical training runs independently to detect regressions and emerging capabilities early.
Manage and grow the evaluations team, including defining the hiring roadmap and supporting long-term development.
Requirements
Proven experience designing, running, and maintaining large, complex evaluation suites for frontier AI systems.
Experience with human and automated evaluation approaches, including AI-based methods.
Research experience designing and implementing evaluations for AI research.
Strong understanding of Python and agentic coding.
At least one year of active people-management experience, with an understanding of team leadership beyond an IC role.
Strong knowledge of variance, confidence intervals, and significance testing, with sound judgment when interpreting numerical results.
Excellent communication skills for explaining evaluation results and their limitations to project leads and research leadership.
First-principles thinking and the ability to derive evaluations for new capabilities.
Culture & Benefits
Hybrid work from Freiburg with regular use of a real office.
Values include scientific rigor, low ego, ambition, shared credit, and kindness.
Compensation includes EUR255K–295K plus equity, depending on profile.
Будьте осторожны
Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →