Назад
Company hidden
10 часов назад

AI Agent Evaluation Expert

Формат работы
remote (только USA)
Тип работы
fulltime
Грейд
senior
Английский
b2
Страна
US/Canada
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
AI Agent Evaluation Expert (AI/Linux/Docker): Designing command-line tasks that test AI agents against real technical problems, with an accent on task authoring, reference solutions, and automated validation. Focus on building reproducible environments, identifying precise failure points, and evaluating workflows across science, engineering, software, security, hardware, business, and creative domains.

Location: Remote; must be based in the US or Canada.

Company

hirify.global connects skilled contributors with technical AI work to evaluate and improve AI agents.

What you will do

  • Turn problems from your research or professional practice into self-contained tasks for AI agents to solve in a Linux terminal.
  • Write clear task briefs and set up environments with the required files and tools.
  • Build reference solutions and automated checks for final results.
  • Identify where technical workflows fail, including issues such as lost precision, dropped data, test leakage, incorrect logic, and conversion errors.
  • Create challenging tasks based on real work across science, engineering, software, machine learning, security, hardware, business, finance, music, and design.

Requirements

  • Hold or be completing a PhD in a relevant field, or have 5+ years of professional experience in one of the listed fields.
  • Have technical or computational experience in the relevant field.
  • Be comfortable working in a Linux terminal, including shell commands, Python or Bash scripting, and package installation.
  • Have used Docker or be able to learn it quickly.
  • Be able to use personal research, projects, code, files, datasets, repositories, or other technical materials as source material for tasks.
  • Have a precise, adversarial approach to finding workflow failures and explaining them clearly.
  • Be honest about uncertainty and write instructions that cannot be misread.
  • Be based in the US or Canada.
  • AI or machine-learning experience is not required outside the machine-learning domain.

Culture & Benefits

  • Remote contribution with a flexible, self-directed schedule.
  • Work on practical AI evaluation tasks designed to improve the safety and usefulness of AI systems.
  • Templates and worked examples are provided for Docker-based task development.
  • Academics and experienced practitioners are welcome.

Hiring process

  • Describe your background and submit an application for review.
  • hirify.global will follow up by email.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →