Senior Manager, Data Quality & Evaluation (AI)

Location: Remote

Company

hirify.global is a global language, data, and AI services company that helps organizations build, evaluate, and improve AI systems.

What you will do

• Design and manage quality frameworks for AI data and evaluation programs, including standards, rubrics, acceptance criteria, KPIs, and scalable workflows.
• Lead calibration, QA sampling, adjudication, reviewer performance tracking, error analysis, and corrective action planning across multilingual and specialized evaluation programs.
• Create reviewer guides, calibration exercises, golden datasets, quality reports, scorecards, error taxonomies, and reusable evaluation assets.
• Partner with Program Management, Supply Chain, Solutions, Sales, Operations, and customer-facing teams to translate requirements into effective quality processes.
• Present data-backed quality readouts, manage escalations, and support pilots, business reviews, and program scale-up discussions.
• Manage and coach Quality Managers, Quality Leads, Quality Specialists, reviewers, and QA contributors while supporting hiring, training, and coverage planning.

Requirements

• 5+ years of experience in quality operations, data operations, AI data services, localization quality, annotation quality, evaluation operations, trust and safety quality, or a related field.
• Experience managing quality programs for complex customer accounts or high-volume operational delivery.
• Strong knowledge of QA methodologies, calibration, sampling, adjudication, error analysis, and performance reporting.
• Experience working cross-functionally with delivery, operations, supply chain, sales, and customer-facing teams.
• Strong analytical, written, verbal communication, and people leadership skills.
• Comfort operating in fast-moving, ambiguous environments where processes are still being built.

Nice to have

• Experience with AI data, RLHF, LLM evaluation, speech/audio evaluation, transcription, coding evaluation, multilingual evaluation, or expert review programs.
• Experience designing rubrics, annotation guidelines, evaluation instructions, reviewer training, calibration workflows, or quality scorecards.
• Experience supporting AI labs, enterprise AI teams, research teams, or technical customers.
• Familiarity with human-in-the-loop workflows, annotation platforms, QA tooling, dashboards, and data labeling operations.
• Experience with expert contributors, linguists, annotators, domain specialists, technical reviewers, or distributed talent networks.

Culture & Benefits

• Remote work in a global AI and language services environment.
• Cross-functional collaboration across Data Services, Program Management, Supply Chain, Solutions, Sales, and Operations.
• Opportunity to build reusable quality systems and shape a growing Data Services quality function.