Senior Software Engineer (AI Evaluation)

Location: Remote, North America only; at least 6 hours of Pacific Time overlap per day required.

Compensation: $100–$150/hour; provide a specific hourly rate expectation.

Company

Turing is an AGI infrastructure company supporting frontier AI labs with training data, evaluations, and engineering talent.

What you will do

• Design, refine, and iterate on rubrics and evaluation criteria for preference data.
• Review and label data, identifying subtle quality issues in AI-generated work.
• Evaluate coding agents across real-world repositories for technical correctness, robustness, and maintainability.
• Build and maintain pipelines and infrastructure for data generation, collection, and evaluation workflows.
• Document findings and recommendations, and collaborate with researchers and engineers to turn technical judgment into scalable processes.

Requirements

• 5+ years of hands-on software engineering experience.
• Strong proficiency in Python, TypeScript/JavaScript, Go, or another major production language.
• Experience working in substantial real-world codebases.
• Strong code-review skills and technical judgment, including the ability to explain whether implementations are correct or how they can be improved.
• Strong written communication.
• Experience using modern LLMs or AI coding tools.

Nice to have

• Experience with LLM evaluation, coding agents, RLHF, preference data, rubric design, or post-training.

Culture & Benefits

• Freelance independent-contractor engagement.
• Remote work limited to North America.
• Availability of 20–40 hours per week.
• Approximately three-month duration with an ASAP start.

Hiring process

• AI interview of approximately 25 minutes.
• Practical code/AI evaluation exercise of approximately 30 minutes, focused on reviewing AI-generated code rather than competitive programming or algorithm puzzles.
• Hiring manager interview of approximately 20 minutes.