Назад
Company hidden
17 часов назад

Software Engineer (AI Agent Evaluation)

Формат работы
remote (только USA)
Тип работы
fulltime
Грейд
senior
Английский
b2
Страна
US/Canada
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Software Engineer (AI Agent Evaluation): Building self-contained Linux-terminal tasks that test AI agents through realistic software engineering failures, with an accent on automated checks, reference solutions, and adversarial edge cases. Focus on designing challenging evaluations involving concurrency, databases, data pipelines, compilers, frontend systems, and CI/CD failures.

Location: Remote; must be based in the US or Canada

Company

hirify.global connects skilled contributors with meaningful AI work by building a community of experts who evaluate how far AI agents can go on real software engineering problems.

What you will do

  • Turn problems from professional or research work into self-contained Tasks for AI agents to solve in a Linux terminal.
  • Write clear task briefs and configure environments with the required files and tools.
  • Build reference solutions and automated checks for final results.
  • Create difficult evaluations based on real failure modes such as race conditions, data loss during migrations, compiler edge cases, network partitions, and flaky CI builds.

Requirements

  • PhD in computer science, software engineering, or a closely related field, completed or in progress, or 5+ years of professional experience in the field.
  • Technical or computational experience in the relevant specialty and comfort working in a Linux terminal.
  • Ability to use shell commands, scripting in Python or Bash, and package installation.
  • Experience with Docker or ability to learn it quickly.
  • Ability to use personal research, projects, code, files, repositories, schemas, pipelines, logs, or other work materials as Task source material.
  • Precise, adversarial approach to identifying workflow failures and explaining them clearly.
  • Ability to write unambiguous instructions and be honest about uncertainty.
  • Must be based in the US or Canada.

Nice to have

  • Expertise in algorithms and data structures; distributed or operating systems; databases and SQL; data engineering; compilers and programming languages; frontend and web; or DevOps, CI/CD, and build systems.

Culture & Benefits

  • Remote contributor work that can be completed on the contributor’s own schedule.
  • Academics and experienced practitioners are equally welcome.
  • No AI or machine-learning background is required.
  • Templates and worked examples are available for Docker-based tasks.

Hiring process

  • Tell hirify.global about yourself; the team will review the application and follow up by email.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →