AI Evaluation Specialist (AI)

Location: Remote — US, Canada, UK, Ireland, Australia, or New Zealand

Company

hirify.global supports an enterprise AI training project focused on improving the quality of AI assistant outputs.

What you will do

• Evaluate AI-generated responses against rubrics covering accuracy, relevance, quality, and guideline adherence.
• Apply consistent, objective judgment across high volumes of examples.
• Identify reasoning gaps, tool-use issues, logical errors, and inconsistencies.
• Provide concise, actionable feedback on strengths and improvement areas.
• Participate in discussions about rubric interpretation and evolving quality standards.
• Maintain evaluation documentation and contribute to process improvements.

Requirements

• Must be located in the US, Canada, UK, Ireland, Australia, or New Zealand.
• Strong analytical and critical-thinking skills with attention to consistency, accuracy, and fairness.
• Ability to analyze complex information and communicate findings clearly in writing.
• Ability to work independently on high-volume evaluation tasks.
• Strong attention to detail and ability to identify subtle quality issues.
• Experience with grading, quality assurance, editorial review, assessment, annotation, or similar analytical work.

Nice to have

• Advanced, daily use of AI assistants such as ChatGPT, Claude, or similar tools.
• Experience with process improvement, rubric development, or quality assessment.
• Experience resolving ambiguous cases collaboratively and improving evaluation criteria.

Culture & Benefits

• Contractor engagement for an enterprise AI training project.
• Remote work within the specified countries.
• Independent work combined with collaboration on ambiguous evaluation cases.
• Opportunity to contribute to AI evaluation best practices and evolving quality standards.