Head of Assessment and Certification Content

TL;DR

Head of Assessment and Certification Content (AI/Assessment Design): Own the end-to-end design and production of assessment and certification content, building AI workflows that generate, review, and continuously refresh defensible questions at scale. Focus on psychometrically sound item design, scoring and standard-setting for validity/reliability/fairness, and evaluating certification performance using learner and employer feedback.

Location: Belgium; London, United Kingdom; Portugal; United States

Company

hirify.global is a platform for learning data and AI skills through hands-on training and expert instruction.

What you will do

• Own the full lifecycle of assessment and certification content: blueprint and item design, review, publication, and ongoing maintenance, while managing deadlines.
• Design psychometrically sound assessments by writing and reviewing items, defining scoring and passing standards, and protecting validity, reliability, and fairness.
• Build and operate AI workflows (prompting, pipelines, and evaluation) that draft, grade, and quality-check items to scale production without losing rigor.
• Source and collaborate with subject-matter experts to keep certifications current with the evolving data and AI landscape.
• Map certifications to the real skills hiring managers screen for and benchmark against the broader certification landscape to guide certification strategy.
• Improve certifications using pass rates, item statistics, and learner and employer feedback.

Requirements

• STEM technical background (e.g., Computer Science, Data Science, AI, Statistics, Engineering) plus experience evaluating technical competency (coding assessments, technical interviews, certification exams, or graded coursework).
• Proven expertise in assessment design: item writing, scoring, standard-setting, and psychometric principles (validity, reliability, fairness).
• Ability to translate hiring-manager skills for data and AI roles into what a certification measures.
• Strong experience with agentic AI systems and AI automation (e.g., Claude Code, Cursor, Replit) and evidence of significantly increasing output with AI.
• Master’s or PhD in a related STEM discipline, or equivalent industry experience with hands-on expertise in data science, machine learning, or software development.

Culture & Benefits

• Individual contributor role focused on owning assessment and certification content end-to-end.
• Cross-functional collaboration with instructors and teams across curriculum, engineering, product, and marketing.
• Emphasis on building AI systems for scalable, high-quality assessment production.
• Continuous improvement driven by performance metrics and feedback from learners and employers.

Hiring process

• Interviews to evaluate assessment design expertise, psychometrics, and experience running technical assessments or certification programs.
• Discussion of AI workflow experience (agentic AI systems, automation, and evaluation) and how it was used to scale content production.
• Review of how you map certification outcomes to real hiring signals and benchmark against the certification landscape.