German-Speaking AI Evaluation Specialist

TL;DR

German-Speaking AI Evaluation Specialist: Assess and improve quality of AI-generated responses to real user queries in German, ensuring accuracy, relevance, and cultural appropriateness. Focus on performing side-by-side analysis of outputs, identifying nuances in meaning, tone, and context, and applying structured annotation guidelines consistently.

Location: Remote (US, Washington State candidates)

Salary: $30.00 to $35.00 USD/hour

Company

Technology solutions firm headquartered in Bellevue, Washington, leveraging cutting-edge technology to unlock value from assets and transform businesses.

What you will do

• Evaluate and compare AI-generated responses in German
• Perform side-by-side analysis of outputs from different AI systems
• Assess responses for accuracy, relevance, clarity, and instruction-following
• Identify nuances in meaning, tone, and cultural context
• Apply structured annotation guidelines consistently
• Work with datasets of real user queries

Requirements

• Native or professional fluency in German
• Strong English reading comprehension
• Analytical thinking and attention to detail
• Experience with structured evaluation or guidelines

Nice to have

• Background in linguistics, translation, or localization
• Experience with data annotation, AI evaluation, or search relevance

Culture & Benefits

• Medical, dental, and vision coverage
• Flexible Spending Account
• 401k program
• Competitive PTO offerings
• Parental Leave
• Opportunities for professional growth and development