Senior Platform Engineer I, AI Evaluation

TL;DR

Senior Platform Engineer I (AI Evaluation): Evolving and extending an internal evaluation framework for assessing the quality of AI-driven experiences with an accent on improving existing eval systems and recognizing opportunities for improvement. Focus on supporting hill-climbing and data science workflows, gathering internal requirements, and developing documentation and training materials for eval-driven development.

Location: Remote (Continental US, Hawaii, or Canada only)

Salary: $137,871 - $172,339 USD annually for candidates based in the United States and $186,306 - $232,883 CAN annually for candidates based in Canada.

Company

hirify.global is a nonprofit organization dedicated to delivering a free, world-class education to anyone, anywhere through its proven learning platform.

What you will do

• Evolve and extend the internal evaluation framework for AI-driven experiences.
• Identify opportunities for improvement within hirify.global's internal evaluation framework.
• Support hill-climbing and data science workflows for AI evaluation.
• Gather internal requirements and obtain buy-in for system changes.
• Develop documentation and training materials for the evaluation framework.
• Help internal teams adopt an eval-driven development process.

Requirements

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, related field, or equivalent professional experience.
• 5 years of Software Engineering experience, including significant work on evaluating generative AI systems or ML model quality.
• Strong programming skills in Go, Python, SQL, and at least one data pipeline framework (e.g., Airflow, Dagster, Prefect).
• Familiarity with the architecture of large language models and their industry-standard APIs.
• Must be based in the Continental US, Hawaii, or Canada.

Culture & Benefits

• Competitive salaries and ample paid time off, including 8 pre-scheduled Wellness Days.
• Remote-first culture with open flexibility and catered to your time zone.
• Generous parental leave.
• Exceptional team that fosters trust and autonomy.
• Opportunity to work on high-impact products defining the future of education.
• 401(k) + 4% matching and comprehensive insurance (medical, dental, vision, life).