Quality Evaluator – English (AI)

Location: Remote from Argentina, Brazil, Chile, Colombia, the Czech Republic, Ecuador, Estonia, France, Italy, Mexico, Paraguay, Peru, Poland, Portugal, Spain, or Uruguay

Compensation: $7–$12 per hour

Company

Creates training and evaluation data for frontier AI models.

What you will do

• Interact with different AI models and tools across a wide range of prompts and topics.
• Compare two or more AI-generated responses side by side.
• Assess accuracy, helpfulness, instruction-following, clarity, and overall response quality.
• Follow detailed rating guidelines and multi-step evaluation workflows.
• Write clear, well-reasoned explanations supporting evaluation decisions.
• Apply evaluation criteria consistently across tasks.

Requirements

• Fluent English with strong reading comprehension and writing skills.
• Extensive hands-on experience using large language models and critically assessing their outputs.
• Strong analytical judgment and the ability to weigh multiple dimensions of response quality.
• Excellent attention to detail and the ability to follow complex instructions carefully.
• Ability to explain evaluation decisions clearly and convincingly.
• Comfort switching between AI models, tools, and multi-step workflows.

Nice to have

• Experience in AI response evaluation, data annotation, or quality assurance.
• Experience in editing, research, fact-checking, or analytical writing.
• Familiarity with evaluation rubrics and side-by-side comparison tasks.

Culture & Benefits

• Independent contractor engagement.
• Part-time commitment of up to three months.
• Fully remote work from the listed countries.