Expert Annotator (AI)

TL;DR

Expert Annotator (AI): Evaluating AI-generated responses for accuracy, coherence, helpfulness, and instruction-following with an accent on identifying logical flaws, hallucinations, and factual errors. Focus on reviewing outputs, ranking responses, writing better answers, and providing structured feedback to train language models.

Remote from anywhere, flexible schedule, no location restrictions

$20-$27+/HR, scales with complexity and performance, weekly payouts

Company

AI training data platform with 100k+ expert annotators and $150M+ paid out.

What you will do

• Assess AI responses for accuracy, coherence, helpfulness, tone, and instruction adherence.
• Review and rank multiple AI outputs side-by-side, explaining strengths and weaknesses.
• Author high-quality responses as gold standard training data.
• Fact-check claims, detect hallucinations, and evaluate logical structure in arguments.
• Rate conversational quality, safety, creative writing, and multi-turn dialogues.

Requirements

• Strong reasoning skills to identify logical gaps, factual errors, and misinterpretations.
• Clear, structured writing for feedback and example responses.
• Attention to detail and analytical rigor.
• Pass a 30-min reasoning assessment (no interview needed).
• Flexible availability for 10–50+ hours/week around other commitments.

Nice to have

• Subject-matter expertise (writing, research, education, analysis).
• Experience in fact-checking, editorial work, or instructional design.

Culture & Benefits

• Full flexibility: no fixed hours, standups, or managers; tasks available 24/7.
• Work on your schedule with per-task payment processed weekly.
• Direct impact on improving leading AI models.
• Join a selective community of experts (writers, lawyers, professors, engineers).

Hiring process

• Complete short reasoning assessment (~30 min) to qualify.
• Get matched to projects based on strengths (no recruiter).
• Start working on tasks and get paid weekly.