Director, Applied AI Assessment Solutions

Location: Fully remote within the United States; candidates near hirify.global offices may choose hybrid work with in-office days on Tuesday and Wednesday. Authorization to work in the United States is required.

Salary: $88,000–$135,000 annually, adjusted based on location, experience, and qualifications.

Company

hirify.global is a self-sustaining nonprofit that develops educational assessments and supports expanded educational and career opportunities.

What you will do

• Lead end-to-end engagements for assessment programs, defining requirements and building AI tools, prompts, and workflows for item development.
• Evaluate AI-generated content for construct alignment, reasoning quality, bias, factual accuracy, and compliance with assessment standards.
• Establish rubrics, benchmarks, and scoring frameworks for systematic AI output evaluation and provide final quality approval.
• Contribute to agentic and multi-agent workflow standards, shared tooling, and enterprise-wide AI enablement roadmaps.
• Design training, documentation, adoption plans, and impact metrics for content teams.
• Experiment with emerging AI models and techniques and share findings through internal communities and external professional meetings.

Requirements

• 7+ years of standardized assessment item-development experience across multiple-choice, free-response, discrete, and set-based items.
• 3+ years of hands-on experience with AI-assisted or agentic techniques, including automated item generation, retrieval-augmented generation, prompt engineering, and context engineering.
• Deep expertise in Reading Comprehension, Writing, Math, or Science assessment.
• Experience with large language model limitations, agentic workflows, rapid prototyping, item banking systems, and digital authoring tools.
• Ability to translate between technical and non-technical stakeholders, lead adoption, manage project plans, and collaborate across functions.
• Ability to travel 6–8 times per year; a master's degree or equivalent education and experience is strongly preferred.

Culture & Benefits

• Mission-driven work focused on expanding educational and career opportunities.
• Supportive, collaborative environment with a learner's mindset and emphasis on continuous improvement.
• Remote work option, with hybrid flexibility for candidates near hirify.global offices.
• Compensation is adjusted by location and based on market data, experience, qualifications, and impact.

Hiring process

• Application review begins immediately and continues until the position is filled, with applications expected to remain open for at least five business days.
• Typical steps include application review, recruiter screening, hiring manager interview, an item review and analysis exercise, panel interview, leadership conversation, and reference checks.