AI Response Labeler / Annotator (Japanese)

Location: Remote within the United States; candidates must be authorized to work in the United States and reside within a U.S. time zone. During the approximately 30-day training and qualification period, work hours are 9:00 a.m. to 5:00 p.m. Pacific Time.

Salary: $38.46–$40.87 USD per hour, with a midpoint of $39.66 USD per hour.

Company

hirify.global is a technology solutions firm connecting strategy and execution across AI, cloud, data, product development, and emerging technology.

What you will do

• Compare AI-generated responses side by side and determine which response better meets the user’s needs.
• Evaluate factual accuracy, relevance, completeness, reasoning, clarity, instruction-following, tone, safety, and overall usefulness.
• Assess English, Japanese, and mixed-language content across questions and answers, search results, file-based tasks, image-based responses, content-generation requests, and single-turn or multi-turn conversations.
• Apply Japanese linguistic and cultural expertise, including terminology, regional conventions, idioms, tone, and cultural context in Japan.
• Document concise, evidence-based rationales and make independent decisions when guidelines do not provide an obvious answer.
• Complete a high volume of evaluations while meeting productivity and quality expectations, generally completing at least 25 tasks per day.

Requirements

• Native-level or professional fluency in Japanese, with deep familiarity with Japanese language conventions and cultural context in Japan.
• Strong English fluency and reading comprehension for complex prompts, AI responses, and detailed English-language annotation guidelines.
• Strong analytical, critical-thinking, written communication, and decision-making skills.
• Ability to evaluate factuality, relevance, reasoning, clarity, instruction-following, cultural appropriateness, and overall usefulness across varied topics and formats.
• Ability to maintain accuracy and consistent judgment during repetitive, detail-oriented work and changing evaluation frameworks.
• Must be authorized to work in the United States and reside within a U.S. time zone.

Nice to have

• Experience with side-by-side labeling, data annotation, comparative content evaluation, or quality assessment.
• Experience evaluating AI-generated responses or contributing to model-quality assessment.
• Experience evaluating search relevance, content quality, factual accuracy, or user-facing digital experiences.
• Experience working with detailed guidelines, rubrics, or structured decision-making frameworks.

Culture & Benefits

• Medical, dental, and vision coverage may be available to eligible employees.
• Flexible Spending Account and 401(k) retirement plan.
• Competitive paid time off and parental leave.
• Professional growth and development opportunities.
• Structured onboarding, calibration, feedback, and ongoing quality-review support.

Hiring process

• Complete a structured onboarding and qualification program before beginning production work.
• Participate in training sessions, guided practice, calibration against quality benchmarks, and a formal qualification review.
• Continue receiving feedback and calibration support after entering production.