Research Engineer - Code Generation & Model Evaluation (AI)

Location: Remote; eligible from the US

Company

hirify.global Human Data pairs experienced engineers with a modeling team to evaluate AI-generated code and software engineering agent outputs.

What you will do

• Run and reproduce generated code outputs in a sandboxed environment.
• Evaluate production code, debugging traces, and developer-facing AI outputs for correctness, style, and edge-case handling.
• Write minimal failing unit tests that demonstrate missed bugs.
• Compare paired solutions, rank them against a rubric, and provide written rationales.
• Tag failure modes such as compile errors, runtime crashes, off-by-one errors, and security issues with severity scores.
• Document recurring code-generation failures and calibrate reviews against gold-standard examples to maintain inter-rater agreement above target.

Requirements

• Strong day-job engineering experience, including the ability to read, run, and debug unfamiliar code.
• Ability to write concise unit tests that capture a single failure mode.
• Familiarity with a testing framework such as pytest, Jest, JUnit, Go test, or RSpec.
• Clear written reasoning that can convince another senior engineer.
• Reliable asynchronous availability for at least 10 hours per week.
• Availability to work as an independent specialist contractor on program-defined task volume and review pacing.
• Eligibility to work from the US.

Nice to have

• Prior code-review or technical-interview-grading experience.
• Open-source contributions or a public portfolio demonstrating production-quality code.
• Experience with the target language’s standard tooling, linters, and idiomatic style guides.
• Familiarity with security-review checklists such as OWASP and CWE, and with application-security patterns.

Culture & Benefits

• Remote independent-contractor arrangement.
• Hourly rate confirmed after the interview process.
• Task volume and review pacing are defined by the program.
• Placement depends on current program demand and reviewer confirmation.

Hiring process

• Submit an application for the specialist cohort.
• Complete human review after intake and confirm profile and eligibility details.
• Receive follow-up if the profile fits a current program.