Senior AI Coding Agent Evaluator

Location: Remote, worldwide

Salary: $100–$200/hour, depending on experience and location

Company

hirify.global connects experienced engineers with projects involving software development and evaluation of modern coding tools.

What you will do

• Evaluate AI-generated coding interactions from start to finish.
• Judge whether responses are useful, correct at a high level, and aligned with strong engineering judgment.
• Assess the quality of explanations, reasoning, preambles, and guidance—not only code syntax.
• Distinguish different levels of response quality and define what excellent interaction looks like.
• Provide clear, opinionated written feedback on what worked, failed, or felt misleading.
• Record short video explanations of evaluation decisions.

Requirements

• Senior, Staff, or Principal-level engineering experience, or equivalent.
• Strong background in TypeScript/JavaScript or Python.
• Hands-on experience with at least one of OpenAI Codex, Claude Code, or Cursor.
• Deep familiarity with modern AI-assisted development workflows.
• Strong written and spoken English at B2 level or above is required.
• Experience with prompt design or evaluation workflows, mentoring senior engineers, or defining engineering standards.

Nice to have

• Experience evaluating code without executing it or reviewing every line.
• Prior experience recording technical video explanations.

Culture & Benefits

• Contract engagement with flexible hours of 10–20 hours per week.
• Projects typically last from approximately two weeks to a few months.
• Ongoing opportunities are offered to evaluators who perform well.
• Start as soon as the take-home evaluation is completed and a project seat is available.

Hiring process

• Complete one take-home evaluation exercise.
• Submit a recorded Loom walkthrough.
• No interview is required.