Systems Engineer (Coding Agent Experience)

Location: Remote; eligible from the US

Salary: $85 per hour

Company

hirify.global Human Data supports AI model evaluation by pairing experienced engineers with modeling teams to assess generated code and software engineering agents.

What you will do

• Run and reproduce generated code outputs in sandboxed environments.
• Evaluate systems engineering review solutions for correctness, style, and edge-case handling.
• Write minimal failing unit tests that demonstrate missed bugs.
• Compare paired solutions, rank them, and provide written rationale tied to the rubric.
• Tag failure modes such as compile errors, runtime crashes, off-by-one errors, and security issues with severity scores.
• Document recurring code-generation failures and calibrate against gold-standard reviews to maintain inter-rater agreement above target.

Requirements

• Strong professional engineering experience, including the ability to read, run, and debug unfamiliar code.
• Comfort writing concise unit tests that capture a single failure mode.
• Familiarity with a testing framework such as pytest, Jest, JUnit, Go test, or RSpec.
• Clear written reasoning that can persuade another senior engineer.
• Reliable asynchronous availability for at least 10 hours per week.

Nice to have

• Prior code-review or technical-interview-grading experience.
• Open-source contributions or a public portfolio demonstrating production-quality code.
• Experience with standard tooling, linters, and idiomatic style guides for the target language.
• Familiarity with OWASP, CWE, security-review checklists, and application security patterns.

Culture & Benefits

• Remote work as an independent specialist contractor.
• Program-defined task volume and review pacing.
• Compensation of $85 per hour.
• Placement depends on current program demand and reviewer confirmation.

Hiring process

• Submit the application form for direct review by the recruiting team.
• Complete human review after intake and confirm profile and eligibility details.
• Placement timing depends on program demand and reviewer confirmation.