AI Evaluation Engineer (Physical Engineering)

Location: Remote in the US or Canada, or San Mateo, California; remote employees work on-site in San Mateo one week out of six. Relocation support is available.

Company

hirify.global is building Archie, an AI engineer agent for physical-world engineering workflows, with the long-term goal of developing engineering superintelligence.

What you will do

• Study how engineers and customers use Archie and identify engineering successes, failures, and workflow gaps.
• Turn real-world observations into actionable evaluation tasks and methodologies.
• Recreate engineering tasks and failure modes in repeatable environments for development teams.
• Design and refine automated judges and human evaluation processes.
• Compare automated judgments with expert human assessments and improve judges when they diverge.
• Work with engineering, product, customers, subject-matter experts, partners, and internal technical teams to prioritize meaningful problems.

Requirements

• Technical background in a physical engineering domain such as power systems, power electronics, mechanical engineering, aerospace, robotics, or a related field.
• Strong understanding of agentic AI systems and how to design and reason about evaluations.
• Experience creating AI evaluation tasks, designing or crafting judges, and comparing automated evaluation with human judgment.
• Ability to work directly with customers, engineers, subject-matter experts, partners, and technical teams in ambiguous workflows.
• Strong product judgment and ability to convert system behavior into concrete evaluation work.
• Ability to work remotely from the US or Canada and attend the San Mateo office one week out of six.

Culture & Benefits

• Work at the intersection of AI and physical engineering.
• Competitive salary and meaningful equity ownership.
• Healthcare, dental, and vision coverage.
• 401(k) match.
• Unlimited paid time off.

Hiring process

• Introductory call lasting 30 minutes.
• Biographical or behavioral interview lasting 45 minutes.
• Technical interview lasting 60 minutes, followed by a 30-minute CEO interview.