AI Quality Analyst (AI)

Location: Remote, with full-time availability in the local time zone and 4 hours of overlap with PST.

Company

hirify.global supports frontier AI research and enterprise AI deployments through high-quality data, training pipelines, and AI research expertise.

What you will do

• Design and execute 1–5-turn conversational prompts using personal context and experiences.
• Evaluate personalized Gemini responses for grounding, integration, helpfulness, and naturalness.
• Identify incorrect personalization, unsupported claims, poor inferences, hallucinations, and forced connections.
• Compare model responses side by side and determine which is more helpful, usable, and enjoyable.
• Write structured rationales referencing specific conversation turns and provide detailed annotations.
• Verify debug information and delete evaluation conversations to maintain data hygiene.

Requirements

• English: Strong written and reading proficiency required.
• Exceptional analytical thinking and attention to detail when evaluating nuanced AI responses.
• Experience designing creative, multi-turn prompts and evaluating personalization quality.
• Excellent written communication, feedback, and collaboration skills.
• Willingness to use a primary personal Google account and enable personal data sources for evaluation.
• BS/BA degree or equivalent experience in policy, law, ethics, linguistics, journalism, computer science, or a related analytical field.

Nice to have

• Experience in data annotation, AI quality evaluation, content moderation, or a related role.

Culture & Benefits

• Remote work in a global, 24-hour operations team.
• Contract engagement requiring at least 4 hours per day and up to 40 hours per week.
• Three-month engagement.
• Desktop or laptop with a reliable internet connection required.

Hiring process

• Shortlisted candidates receive a job interest form.
• After profile review, candidates complete an assessment within 24 hours.
• Successful candidates discuss pre-onboarding requirements.