Principal Data Scientist (Information Retrieval)

TL;DR

Principal Data Scientist (Information Retrieval): Lead the development and evaluation of offline search metrics for Bing, driving data-driven insights through rigorous offline analyses with an accent on API design, database performance, and scalability. Focus on leveraging state-of-the-art BigData technologies to sample, process, and analyze vast search logs, formulating and validating hypotheses regarding search relevance, and delivering actionable findings to leadership.

Location: Mountain View, United States. Employees within a 50-mile commute of a designated Microsoft office in the U.S. or 25-mile commute of a non-U.S. country-specific location are expected to work from the office at least four days per week.

Salary: USD $139,900 – $274,800 per year (typical U.S. base pay range).

Company

hirify.global (MAI) is a leading technology company empowering individuals and organizations worldwide to achieve more, fostering a culture of inclusion, growth, and innovation.

What you will do

• Sample large, representative datasets from extensive search log repositories using BigData Map-Reduce frameworks.
• Architect and implement scalable data processing and analysis pipelines for offline metric computation.
• Formulate, test, and validate hypotheses regarding search result quality using advanced statistical methods and machine learning models.
• Design labeling protocols and manage crowd judges for high-fidelity data annotation and validation.
• Adapt evaluation workflows to incorporate LLM-as-a-judge and fine-tuned LLMs.
• Develop and deliver custom reports, visualizations, and presentations to communicate insights to senior leadership.
• Collaborate across multidisciplinary teams to extend offline metric methodologies.

Requirements

• Experience: 10+ years of data science experience with a Bachelor’s degree, 7+ years with a Master’s, or 5+ years with a Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field.
• Extensive experience in Information Retrieval, including designing, evaluating, and optimizing search and ranking algorithms.
• Demonstrated ability to develop and validate metrics for Information Retrieval systems, ensuring robust measurement of relevance, accuracy, and user satisfaction.
• Expertise in crowdscience methodologies, including designing and running large-scale crowdsourcing experiments for data annotation and model evaluation.
• Proven track record in architecting and deploying large-scale data pipelines for real-time and batch processing of heterogeneous data sources.
• Solid background in experimental design, statistical analysis, and A/B testing for data-driven product improvements.
• Ability to lead cross-functional teams and mentor junior scientists in best practices for data science and machine learning.

Nice to have

• Familiarity with Python, T-SQL, and HTML/JS for dashboarding.

Culture & Benefits

• Empowering every person and organization on the planet to achieve more.
• Culture of inclusion, growth, and innovation, built on values of respect, integrity, and accountability.
• Opportunity to collaborate extensively across engineering, product, and research teams.
• Directly influence Bing’s search quality and relevance and drive innovation.
• Shape product strategy and have a broad impact across the Bing organization.
• Specific benefits and pay information can be found on the Microsoft careers website.