Вакансия из Hirify Global, списка международных tech-компаний Для мэтча и отклика нужен Plus
Пожаловаться
Зарплата и рынок
ЗП не указанаразвернуть
На похожих ролях по рынку
$83к$125кмедиана$144к
41 в выборке
На СНГ рынке: $4.3к/мес ($3.6к - $5.4к)
91
Крутая вакансия
развернуть
Здесь чёткая ответственность за важное направление оценки ИИ и серьёзные инженерные задачи для продакшена, включая безопасность; условия оплаты не указаны.
Актуальные технологии
Чёткая зона ответственности
Перспективная область
?Высокие ожидания
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Подключить Plus
Описание вакансии
Текст:
Hirify AI/Оригинал
TL;DR
Senior AI Engineer - Agentic AI Evaluation (Agentic AI/Python): Building the evaluation engine for tool-using, multi-step agents with an accent on stateful behavior, evaluator integrity, security, and production reliability. Focus on designing trustworthy benchmarks, testing adversarial agent behavior, translating AI governance requirements into evaluation logic, and extending assessment to autonomous systems.
Location: Hybrid in Munich, Germany or Singapore
Company
hirify.global is an independent third-party AI assurance company building software and evaluations for trustworthy deployment of computer vision, LLM, vision-language, agentic, and autonomous AI systems in government, defence, and regulated commercial sectors.
What you will do
Own and build the agentic-evaluation engine for scoring intent resolution, tool selection and parameterisation, task adherence, planning, tool-call ordering, response completeness, and human handoff.
Design harnesses for stateful, multi-turn agents to expose behavioral drift, degradation, and safety issues across long sessions.
Build trustworthy evaluations by isolating runners, checking semantics, treating crashed or incomplete tasks as failures, auditing ground truth, and testing the evaluation setup itself.
Partner on red-teaming and adversarial evaluation covering tool-chain exploits, RBAC and permission-inheritance failures, and multi-turn safety erosion.
Translate EU AI Act, NIST AI RMF, and ISO 42001 requirements into testable evaluation logic and close the lab-to-production assurance gap.
Set the technical direction through spikes, ADRs, code review, and future extensions for embodied and autonomous systems.
Requirements
5+ years of professional software engineering experience, including shipping and operating production systems in Python; prototype-only experience is insufficient.
Demonstrated depth in agentic AI, including building, orchestrating, or rigorously evaluating LLM agents and understanding tool calling, memory, multi-step control flow, MCP, and A2A beyond tutorial level.
Knowledge of execution-based benchmarks including GAIA, τ²-bench, OSWorld, SWE-bench Verified, and WebArena, including their limitations and failure modes.
Strong engineering judgment with the ability to scope ambiguous research-adjacent problems and drive them to shipped, tested, observable outcomes independently.
Hands-on technical depth combined with clear documentation, communication, and effective handover.
Clear written and verbal communication and comfort working toward concrete quarterly outcomes across a distributed Singapore-Europe team.
Nice to have
Agentic security or red-teaming experience, including adversarial testing, jailbreaks, tool-chain exploits, or agent identity and on-behalf-of authorisation.
AI evaluation, testing, or assurance tooling experience, including LLM/RAG, computer vision, LLM-as-judge pipelines, and their failure modes.
Familiarity with AI governance frameworks and translating principles into engineering requirements.
Exposure to embodied or autonomous AI, including vision-language-action models, simulators, or world models.
Experience with data-intensive pipelines using Parquet, Iceberg, DuckDB, or similar technologies.
Experience with container-based or serverless evaluation frameworks, Kubernetes/Helm, or GPU/CUDA infrastructure.
Experience in regulated, safety-critical, or defence contexts where evidence, auditability, and reproducibility were first-class requirements.
Culture & Benefits
Hands-on individual-contributor role with domain ownership and no line management responsibility.
Close partnership with Product, governance analysts, and an Engineering Manager responsible for the evaluation engine.
High, transparent engineering bar focused on production-grade, tested, observable, maintainable code.
Distributed collaboration across Singapore and Europe with emphasis on clear communication, documented decisions, independent operation, and effective handover.
Equal opportunity workplace supporting diverse cultures, perspectives, skills, and experiences.
Будьте осторожны
Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →