Search Quality Analyst

TL;DR

Search Quality Analyst: Develop data-driven insights and design A/B tests to improve advanced search technologies with an accent on user behavior analysis, metric design, and data pipeline construction. Focus on designing online metrics, analyzing experiments, and integrating human and LLM feedback for ranking quality.

Location: Belgrade, London, Berlin

Company

Perplexity is focused on developing advanced search technologies leveraging data analysis and machine learning.

What you will do

• Develop insights from user behavior to inform product roadmap and adoption
• Design, run, and analyze A/B tests to validate hypotheses
• Create metrics and dashboards for tracking search quality
• Design data pipelines for ranking quality using human and LLM feedback

Requirements

• 4+ years experience as a data analyst or related role
• Experience with search-related products and A/B experiment analysis
• Strong Python and SQL skills
• Proficiency with Business Intelligence tools
• Deep knowledge of statistics

Nice to have

• Experience with Apache Spark and Databricks
• Experience developing LLM-as-a-judge systems