В Company hidden AI это чётко очерченная руководящая роль в сфере оценки передовых моделей, но указанная зарплата ниже медианы сопоставимых вакансий.
Ниже рынка
Современный стек
Передовой ИИ
Понятная роль
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Подключить Plus
Описание вакансии
Текст:
Hirify AI/Оригинал
TL;DR
AI Evaluation Systems Engineering Lead (AI): Building evaluation infrastructure that converts model checkpoints and deployed models into reliable, reproducible results with an accent on benchmark configuration, inference, grading, aggregation, and large-scale execution. Focus on improving GPU efficiency and evaluation throughput, diagnosing distributed-system failures, ensuring operational reliability, and leading the engineering team through complex infrastructure delivery.
Location: Mountain View, California, United States
Base pay: USD $142,800–$274,800 per year across the U.S.; USD $188,000–$304,200 per year in the San Francisco Bay Area and New York City metropolitan area for IC5. IC6 base pay: USD $165,600–$296,400 per year across the U.S.; USD $220,800–$331,200 per year in those metropolitan areas.
Company
hirify.global AI develops frontier AI models and the infrastructure used to evaluate, improve, and release them reliably.
What you will do
Set the Eval Systems team’s architecture, technical roadmap, delivery priorities, and long-term reliability strategy.
Build end-to-end evaluation infrastructure for benchmark configuration, model inference, grading, aggregation, and results publication.
Accelerate research iteration by improving evaluation throughput, scheduling, turnaround time, and GPU efficiency.
Contribute code, lead design reviews, and debug problems across evaluation, inference, and distributed execution.
Hire, mentor, and develop engineers while aligning research and engineering partners on priorities, dependencies, and tradeoffs.
Requirements
Bachelor’s degree in Computer Science or a related technical discipline and 6+ years of technical engineering experience with coding in C, C++, C#, Java, JavaScript, or Python, or equivalent experience.
Nice to have
Bachelor’s degree in Computer Science or a related technical field and 10+ years of technical engineering experience.
Experience building and operating production distributed systems, infrastructure platforms, or ML systems.
Technical leadership experience setting direction and delivering complex projects through a team.
Strong Python programming skills and experience contributing to large shared codebases.
Experience diagnosing distributed-system failures and improving reliability, performance, and resource efficiency.
Production operations experience, including monitoring, incident response, and on-call practices.
Experience translating evolving requirements into engineering priorities and partnering with research and engineering teams.
Experience with LLM evaluation, benchmarking, model-based grading, reinforcement learning infrastructure, GPU-backed inference, distributed training, or large-scale compute workloads.
Familiarity with Ray, Kubernetes, Slurm, or other distributed execution frameworks and schedulers.
Experience improving experimental reproducibility, validating metrics, or investigating differences between evaluation and serving environments.
Experience simplifying complex infrastructure and reducing operational burden through automation and systematic reliability improvements.
Culture & Benefits
Hands-on engineering leadership role combining architecture, implementation, team development, and cross-functional collaboration.
Focus on clear ownership, collaboration, technical excellence, inclusive teamwork, and dependable operational practices.
Benefits and other compensation may be available.
The position remains open for at least 5 days, with applications accepted on an ongoing basis until filled.
Будьте осторожны
Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →