Principal Data Or Applied Scientists For Search Relevance Metrics (AI)

TL;DR

Principal Data or Applied Scientist (AI): Leading the development and evaluation of offline search metrics for Bing with an accent on search quality measurement and data-driven insights. Focus on shaping product strategy and enabling data-informed decision-making at the highest levels.

Location: Must live within a 50- mile commute of a designated hirify.global office in the U.S. or 25-mile commute of a non-U.S., country-specific location and are expected to work from the office at least four days per week.

Salary: USD $139,900 – $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 – $304,200 per year.

Company

hirify.global’s mission is to empower every person and every organization on the planet to achieve more.

What you will do

• Sample large, representative datasets from extensive search log repositories utilizing BigData Map-Reduce frameworks and distributed data platforms.
• Architect and implement scalable data processing and analysis pipelines for offline metric computation, leveraging modern data engineering best practices.
• Formulate, test, and validate hypotheses regarding search result quality using advanced statistical methods and machine learning models.
• Design labeling protocols and manage trained crowd judges and auditors for high-fidelity data annotation and validation of search results.
• Adapt traditional evaluation workflows to incorporate LLM-as-a-judge and fine-tuned LLMs, ensuring robust and scalable quality assessments.
• Develop and deliver custom reports, visualizations, and presentations to communicate insights and recommendations to senior leadership.

Requirements

• Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5+ years data-science experience OR Master’s Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 7+ years data-science experience OR Bachelor’s Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 10+ years data science experience OR equivalent experience.
• Extensive experience in Information Retrieval, including designing, evaluating, and optimizing search and ranking algorithms.
• Demonstrated ability to develop and validate metrics for Information Retrieval systems, ensuring robust measurement of relevance, accuracy, and user satisfaction.
• Expertise in crowdscience methodologies, including designing and running large-scale crowdsourcing experiments for data annotation and model evaluation.
• Proven track record in architecting and deploying large-scale data pipelines for real-time and batch processing of heterogeneous data sources.
• Solid background in experimental design, statistical analysis, and A/B testing for data-driven product improvements.

Nice to have

• Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 8+ years data-science experience OR Master’s Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 10+ years data-science experience OR Bachelor’s Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 12+ years data-science experience OR equivalent experience.
• Ability to lead cross-functional teams and mentor junior scientists in best practices for data science and machine learning.
• Ability to work independently, solid collaboration and communication skills
• Familiarity with Python, T-SQL, (HTML/JS for dashboarding).

Culture & Benefits

• Foster a culture of inclusion, growth, and innovation, built on values of respect, integrity, and accountability.
• Passionate about driving meaningful impact, solving complex problems, and contributing to a growing organization.