Principal Data Scientist (AI, Search)

TL;DR

Principal Data Scientist (AI, Search): Leading the development and evaluation of offline search metrics for Bing with an accent on driving data-driven insights through rigorous offline analyses. Focus on leveraging state-of-the-art BigData technologies, formulating and validating hypotheses regarding search relevance, and shaping future search quality at scale.

Location: Redmond, United States. This is a hybrid role; employees within a 50-mile commute of a designated Microsoft office in the U.S. are expected to work from the office at least four days per week.

Salary: USD $139,900–$274,800 per year (general U.S.), or USD $188,000–$304,200 per year (San Francisco Bay area and New York City metropolitan area).

Company

hirify.global aims to empower every person and organization on the planet to achieve more, fostering a culture of inclusion, growth, and innovation.

What you will do

• Sample large, representative datasets from extensive search log repositories utilizing BigData Map-Reduce frameworks.
• Architect and implement scalable data processing and analysis pipelines for offline metric computation.
• Formulate, test, and validate hypotheses regarding search result quality using advanced statistical methods and machine learning models.
• Design labeling protocols and manage trained crowd judges and auditors for high-fidelity data annotation.
• Adapt traditional evaluation workflows to incorporate LLM-as-a-judge and fine-tuned LLMs.
• Develop and deliver custom reports, visualizations, and presentations to senior leadership.

Requirements

• Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, or Computer Science AND 5+ years of data-science experience (or Master’s with 7+, or Bachelor’s with 10+ years).
• Extensive experience in Information Retrieval, including designing, evaluating, and optimizing search and ranking algorithms.
• Demonstrated ability to develop and validate metrics for Information Retrieval systems, ensuring robust measurement.
• Expertise in crowdscience methodologies, including designing and running large-scale crowdsourcing experiments for data annotation.
• Proven track record in architecting and deploying large-scale data pipelines for real-time and batch processing.
• Solid background in experimental design, statistical analysis, and A/B testing for data-driven product improvements.

Nice to have

• Ability to lead cross-functional teams and mentor junior scientists in best practices.
• Ability to work independently with solid collaboration and communication skills.
• Familiarity with Python, T-SQL, and HTML/JS for dashboarding.

Culture & Benefits

• A culture of inclusion, growth, and innovation, built on values of respect, integrity, and accountability.
• Opportunity to directly influence Bing’s search quality and relevance, driving innovation in offline metric development.
• Enable data-informed decision-making at the highest levels and shape product strategy.
• Collaborate extensively across engineering, product, and research teams.
• Have a broad impact across the Bing organization, fostering continuous improvement.
• Access to a comprehensive benefits package.