AI Evaluation Lead (AI)

TL;DR

AI Evaluation Lead (AI): Defining and executing strategies for evaluating AI-powered features and LLMs with an accent on developing rigorous evaluation benchmarks and methodologies. Focus on managing teams, mentoring, and communicating findings across the organization.

Location: Amsterdam, Netherlands; Berlin, Germany; Limassol, Cyprus; London, United Kingdom; Munich, Germany; Paphos, Cyprus; Prague, Czech Republic; Warsaw, Poland; Yerevan, Armenia

Company

hirify.global develops robust developer tools and focuses on integrating advanced AI capabilities into their products.

What you will do

• Design and develop evaluation benchmarks for AI features and LLMs.
• Manage the team, prioritize tasks, and mentor teammates.
• Define evaluation methodology for open-source models.
• Communicate findings and best practices across the organization.

Requirements

• Expertise in evaluating generative AI methods.
• Strong understanding of statistics and data analysis.
• Excellent management and communication skills.
• Practical experience with Python and evaluation frameworks.
• Attention to detail.