AI Personalization Evaluators

Location: Remote — Australia only; 4 hours of overlap with PST required.

Company

Boutique consultancy specializing in IT professional services and engineering talent solutions.

What you will do

• Evaluate AI-generated responses using personalized prompts and information retrieved from connected Google applications.
• Assess response relevance, accuracy, helpfulness, and personalization.
• Identify incorrect personalization, unsupported assumptions, irrelevant recommendations, inconsistencies, and subtle quality issues.
• Compare multiple AI-generated responses and evaluate differences in user experience.
• Provide clear, detailed feedback to support AI model improvements while following project guidelines.
• Maintain data privacy and confidentiality throughout the project.

Requirements

• Currently based in Australia.
• Willing and able to connect relevant Google applications to Gemini, subject to consent and permissions.
• Active use of Gmail, Google Calendar, Google Photos, Google Drive, or other Google applications with sufficient personal data or history for personalized retrieval evaluations.
• Strong analytical thinking, attention to detail, and excellent written English.
• Ability to work independently remotely with a reliable desktop or laptop and internet connection.
• Bachelor’s degree or equivalent practical experience in any field.

Culture & Benefits

• Contractor engagement for up to 16 weeks.
• Minimum commitment of 4 hours per day, up to 40 hours per week.
• Remote work with required PST time overlap.
• Project onboarding includes consent and privacy requirements.

Hiring process

• Shortlisted candidates receive a Job Interest Form.
• Selected candidates are contacted about consent and onboarding requirements.