Korean-Speaking AI Evaluation Specialist (AI)

TL;DR

Korean-Speaking AI Evaluation Specialist (AI Evaluation): Assessing and improving the quality of AI-generated responses in Korean with an accent on accuracy, cultural relevance, and instruction-following. Focus on side-by-side analysis of AI outputs and applying structured annotation guidelines to improve LLM communication.

Location: Remote (Must be based in the USA)

Salary: $30.00 - $35.00 per hour

Company

Technology solutions firm helping organizations unlock value from existing assets by leveraging cutting-edge technology to create new revenue streams.

What you will do

• Evaluate and compare AI-generated responses in Korean for accuracy and relevance.
• Perform side-by-side analysis of outputs from different AI systems.
• Assess responses for clarity and strict adherence to provided instructions.
• Identify nuances in meaning, tone, and cultural context.
• Apply structured annotation guidelines consistently to datasets of real user queries.

Requirements

• Native or professional fluency in Korean.
• Strong English reading comprehension.
• Analytical thinking and high attention to detail.
• Experience with structured evaluation or guidelines.
• Must be based in the United States (indicated by 401k and FSA benefits).

Nice to have

• Background in linguistics, translation, or localization.
• Experience with data annotation, AI evaluation, or search relevance.

Culture & Benefits

• Medical, dental, and vision coverage.
• 401k program and Flexible Spending Account (FSA).
• Competitive PTO offerings and Parental Leave.
• Opportunities for professional growth and development.