Director, Search & AI Evaluation (AI)

TL;DR

Director, Search & AI Evaluation (AI): Leading the strategic direction and technical vision for search relevance, retrieval optimization, and AI quality measurement systems with an accent on RAG architectures and large-scale evaluation frameworks. Focus on building scalable evaluation methodologies, mentoring high-performing data science teams, and driving evidence-based decision-making across AI-powered discovery platforms.

Location: London Wall

Company

hirify.global is a global leader in information and analytics, combining quality content and vast data sets to support science, research, and healthcare outcomes.

What you will do

• Define and drive the long-term strategy for search relevance, retrieval evaluation, and AI system quality.
• Lead and mentor a multidisciplinary team of Senior Data Scientists and Analysts.
• Operationalize evaluation frameworks for search and generative AI, including IR metrics and LLM faithfulness.
• Partner with Product, Engineering, and UX teams to align AI investments with business priorities.
• Establish best practices for offline benchmarking, online experimentation, and reproducible evaluation workflows.
• Translate complex technical findings into strategic recommendations for executive leadership.

Requirements

• Master’s or PhD in Computer Science, Data Science, Machine Learning, NLP, or a related quantitative field.
• Significant experience leading and scaling high-performing technical teams in complex organizations.
• Deep expertise in search relevance, ranking systems, information retrieval, and RAG.
• Strong experience with vector retrieval, hybrid search architectures, and LLM evaluation methodologies.
• Advanced programming skills in Python and familiarity with modern AI/ML frameworks like PyTorch, Hugging Face, and LangChain.
• Strong understanding of statistical analysis, experimentation design, and evaluation science.

Nice to have

• Experience building evaluation systems for LLM-powered applications and AI assistants.
• Familiarity with scientific, biomedical, or scholarly datasets.
• Exposure to MLOps, AI governance, and production ML systems.
• Publications or applied research contributions in NLP, IR, or generative AI.

Culture & Benefits

• Flexible working hours to support work-life balance.
• Comprehensive pension plan and generous vacation entitlement.
• Support for wellbeing through initiatives, shared parental leave, and sabbatical options.
• Personal Choice budget and various employee discounts.
• Access to internal communities and networks for professional growth.