Principal Data Scientist (AI)

TL;DR

Principal Data Scientist (AI): Leading the development and evaluation of offline search metrics for Bing with an accent on search quality measurement and large-scale data analysis. Focus on architecting scalable pipelines, leveraging LLM-based evaluation methodologies, and driving data-informed product strategy.

Location: Based in Redmond, USA; hybrid work model requires presence in office 4 days per week if within 50-mile commute.

Salary: $139,900 – $304,200 per year.

Company

A global leader in technology empowering people and organizations to achieve more through innovation, growth, and inclusive culture.

What you will do

• Architect and implement scalable data processing pipelines for offline search metric computation.
• Formulate, test, and validate hypotheses regarding search relevance using advanced statistical methods and machine learning models.
• Design labeling protocols and manage crowd judges for high-fidelity data annotation.
• Adapt evaluation workflows to incorporate LLM-as-a-judge techniques for scalable quality assessments.
• Communicate actionable insights and strategic recommendations to senior leadership via custom reports and visualizations.
• Collaborate across multidisciplinary engineering and product teams to drive search quality innovation.

Requirements

• PhD, Master’s, or Bachelor’s degree in Data Science, Mathematics, Statistics, Computer Science, or related field.
• 5-10+ years of relevant data science experience depending on degree level.
• Expertise in Information Retrieval, including search and ranking algorithm evaluation.
• Strong background in experimental design, statistical analysis, and A/B testing.
• Proven ability to architect large-scale data pipelines for heterogeneous data sources.
• Proficiency in Python and T-SQL.

Nice to have

• Experience with crowdscience methodologies and large-scale crowdsourcing experiments.
• Background in mentoring junior scientists.
• Familiarity with web technologies for dashboarding.

Culture & Benefits

• Comprehensive health, dental, and vision insurance.
• Commitment to an inclusive, respectful, and accountable work culture.
• Opportunities to solve high-impact problems at global scale.
• Professional growth and development programs.