Senior Software Engineer Expert (AI Evaluation)

Location: Remote, US-eligible; eligible from the United States.

Salary: $80–$130 per hour.

Company

hirify.global pairs experienced engineers with a modeling team to evaluate production code and developer-facing AI outputs against real-world correctness standards.

What you will do

• Run and reproduce generated code outputs in a sandboxed environment.
• Grade software engineering review solutions for correctness, style, and edge-case handling.
• Write minimal failing unit tests that demonstrate missed bugs.
• Compare paired solutions, rank them, and provide written rationale tied to the rubric.
• Tag failure modes with severity scores and document recurring code-generation failures.
• Calibrate against gold-standard reviews to maintain inter-rater agreement above target.

Requirements

• Strong day-job engineering experience with the ability to read, run, and debug unfamiliar code.
• Comfort writing concise unit tests that capture a single failure mode.
• Familiarity with a testing framework such as pytest, Jest, JUnit, Go test, or RSpec.
• Clear written reasoning that can convince another senior engineer.
• Reliable asynchronous availability for at least 10 hours per week.

Nice to have

• Prior code-review or technical-interview-grading experience.
• Open-source contributions or a public portfolio demonstrating production-quality code.
• Experience with standard tooling, linters, and idiomatic style guides for the target language.
• Familiarity with security-review checklists, OWASP, CWE, and AppSec patterns.

Culture & Benefits

• Independent specialist contractor arrangement.
• Remote work with asynchronous availability.
• Program-defined task volume and review pacing.
• Placement depends on current program demand and reviewer confirmation.

Hiring process

• Confirm profile and eligibility details.
• Attach the role deliberately during specialist intake.
• Receive follow-up from the recruiting team if the profile fits a current program.