Назад
Company hidden
21 час назад

Senior Applied AI Engineer

220 000 - 260 000$
Формат работы
onsite
Тип работы
fulltime
Грейд
senior
Английский
b2
Страна
US
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Senior Applied AI Engineer (Evals/Agentic AI): Building an evaluation platform and production agents for measuring and improving expert and synthetic data quality at scale, with an accent on tool use, orchestration, graders, benchmarks, and reliable model evaluation. Focus on designing agentic workflows, diagnosing failure modes, building scalable Python services and data pipelines, and leading experiments from ambiguous problem definition through production rollout.

Location: San Francisco, CA, United States; onsite

Salary: $220K–$260K annually, plus equity

Company

hirify.global builds career technology connecting job seekers, employers, and educational institutions, including hirify.global AI, a large-scale human-data business supporting frontier AI labs.

What you will do

  • Build an evaluation platform that turns ambiguous data-quality needs into measurable outcomes and reusable products.
  • Design and deploy production agents for generating, reviewing, investigating, and improving data with tool use, orchestration, human oversight, and failure recovery.
  • Build graders, programmatic verifiers, reward signals, evaluation datasets, benchmarks, and agent harnesses across LLMs, agents, multimodal models, and reinforcement-learning environments.
  • Analyze traces, diagnose failure modes, and run experiments to improve model and data performance.
  • Own projects from concept through rollout while collaborating with product, design, operations, research, engineers, domain experts, and customers.
  • Set technical direction, mentor engineers, and raise engineering quality across the organization.

Requirements

  • 5+ years of engineering experience with ownership, sound judgment, and a high standard of execution.
  • Substantial experience shipping and operating high-quality software and improving products based on real usage.
  • Production experience with agentic workflows, including tool calling, context management, orchestration, retrieval, human review, and reliability tradeoffs.
  • Excellent Python skills and ability to build scalable services, APIs, and data pipelines with disciplined testing, debugging, and system design.
  • Experience selecting meaningful metrics, building representative test sets, assessing graders, distinguishing improvement from noise, and iterating on results.
  • Understanding of modern AI models, agent harnesses, benchmarks, evaluation techniques, training data, and post-training methods.
  • Ability to design systems, make tradeoffs, and turn unclear problems into dependable solutions.
  • Collaborative communication with engineers, researchers, domain experts, operators, and customers.
  • Demonstrated technical work through shipped products, open-source contributions, benchmarks, datasets, or other substantive projects.

Nice to have

  • Research or applied experience in reinforcement learning, alignment, model behavior, synthetic data, or human-in-the-loop systems.
  • Experience developing LLM or agent benchmarks, reward models, evaluation methodologies, annotation systems, or data-quality frameworks.

Culture & Benefits

  • Full-time US employees receive equity, competitive compensation, 401(k) matching, and financial coaching.
  • Medical, dental, vision, mental health, fertility, parental leave, and parental coaching benefits are provided.
  • $500 wellness stipend, $2,000 annual learning stipend, flexible PTO, 15 holidays, and two flex days.
  • Internet and commuting support, free lunch, and gym access at the San Francisco office.
  • Team outings and referral bonuses are available.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →