Japanese-Speaking AI Evaluation Specialist (AI)

TL;DR

AI Evaluation Specialist (Japanese/AI): Assessing and improving the quality of AI-generated responses in Japanese with an accent on accuracy, relevance, and cultural appropriateness. Focus on performing side-by-side analysis of AI outputs and applying structured annotation guidelines to real user queries.

Location: Remote (Must be based in the US)

Salary: $30.00 – $35.00 USD/hour

Company

hirify.global is a technology solutions firm that helps organizations unlock value from existing assets by leveraging cutting-edge technology to create new revenue streams.

What you will do

• Evaluate and compare AI-generated responses in Japanese for accuracy and clarity.
• Perform side-by-side analysis of outputs from different AI systems.
• Assess responses for relevance and strict adherence to instructions.
• Identify nuances in meaning, tone, and cultural context.
• Apply structured annotation guidelines consistently across datasets of real user queries.

Requirements

• Native or professional fluency in Japanese
• Strong English reading comprehension
• Analytical thinking and high attention to detail.
• Experience working with structured evaluation or guidelines.

Nice to have

• Background in linguistics, translation, or localization.
• Experience with data annotation, AI evaluation, or search relevance.

Culture & Benefits

• Medical, dental, and vision coverage.
• 401k program and Flexible Spending Account.
• Competitive PTO offerings and Parental Leave.
• Opportunities for professional growth and development within a diverse team.