AI Evaluation Specialist

Location: Remote — United States, Canada, United Kingdom, Ireland, Australia, or New Zealand

Company

hirify.global supports an enterprise AI training project focused on improving the quality of AI assistant outputs.

What you will do

• Evaluate AI-generated responses against rubrics covering accuracy, relevance, quality, and guideline adherence.
• Apply consistent and objective judgment across high volumes of examples.
• Identify reasoning gaps, tool-use issues, logical errors, and inconsistencies.
• Provide concise, actionable feedback on strengths and improvement areas.
• Participate in discussions about rubric interpretation and evolving quality standards.
• Maintain evaluation documentation and contribute to process improvements.

Requirements

• Experience in grading, quality assurance, editorial review, assessment, annotation, or similar analytical work.
• Advanced daily use of AI assistants such as ChatGPT, Claude, or similar tools.
• Strong analytical, critical-thinking, written communication, and attention-to-detail skills.
• Ability to work independently on high-volume evaluation tasks.
• Ability to identify subtle quality issues and apply standards consistently, accurately, and fairly.
• Must be located in the United States, Canada, United Kingdom, Ireland, Australia, or New Zealand.

Nice to have

• Experience with process improvement, rubric development, or quality assessment.
• Collaborative experience resolving ambiguous cases and improving evaluation criteria.

Culture & Benefits

• Remote contractor role.
• Independent work on an enterprise AI training project.
• Opportunities to contribute to evaluation best practices and evolving quality standards.