Principal Data Scientist (AI)
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Location: Remote within the United States; U.S. employees may live in any of the 50 states, with limited exceptions.
Base salary: $182,100–$290,900 annually in specified states; $173,000–$276,400 annually in another specified group of states. Actual pay varies by experience, performance, and location; equity awards may also apply.
Company
is a U.S. real estate platform helping people buy, sell, finance, and rent homes, with foundational technology platforms supporting consumer, partner, and employee experiences.
What you will do
- Define and operationalize measurement strategies for the Connections Platform and broader Data & Infrastructure Platform, from raw events to executive dashboards and causal attribution.
- Design rigorous, repeatable evaluations for LLM-based and agentic systems, including LLM-as-a-judge and tooling such as Braintrust.
- Lead A/B testing and quasi-experimental causal inference using methods such as switchbacks, difference-in-differences, synthetic control, and interrupted time series.
- Define meaningful metrics, leading indicators, quality signals, and impact measures while challenging vanity metrics and separating adoption from genuine outcomes.
- Partner with engineers on event taxonomies, schemas, pipelines, instrumentation, dashboards, and trusted measurement data.
- Translate complex measurement findings and trade-offs for VP and C-level partners and help establish platform-as-a-product measurement practices.
Requirements
- 10+ years of experience in Data Science, UX Research, or Applied Science, including meaningful experience measuring products where the customer is a developer, internal team, or platform.
- Experience evaluating complex user-facing systems where quality is difficult to define, using labeled test sets, human review, or LLM-as-a-judge.
- Production experience designing and analyzing A/B tests and causal inference methods, with the judgment to select appropriate methods and state their assumptions.
- Experience designing primary metrics, KPIs, OKR definitions, and leading and lagging indicator systems from scratch.
- Ability to design event taxonomies, debug methodology, challenge schemas, and build reliable instrumentation.
- Ability to write Python and SQL, model data, and prototype evaluations and dashboards independently.
- Ability to communicate rigorous measurement narratives and assumptions to VP and Senior Director stakeholders.
- Builder-oriented individual contributor who improves the rigor and repeatability of platform data science.
Nice to have
- PhD with 4+ years, MS with 6+ years, or BS with 8+ years of experience in Statistics, Econometrics, Data Science, User Experience Research, Machine Learning, Operations Research, or a related quantitative field.
- Experience measuring production conversational, LLM-based, or agentic systems.
- Experience with modern evaluation methods, offline and online evaluation harnesses, and LLM-as-a-judge.
- Experience measuring the impact and ROI of AI or agentic tooling on productivity, quality, and business outcomes.
- Experience with Platform-as-a-Product models and internal teams as customers.
- Experience with difference-in-differences, synthetic control, interrupted time series, switchback, or interleaved designs.
Culture & Benefits
- Remote work without a permanent corporate office, from an identified physical location within the United States.
- Cloud HQ supports distributed work and flexibility in choosing a productive work location.
- Full-time employment with competitive base salary and eligibility for equity awards.
- Inclusive equal-opportunity workplace with accommodations available for applicants with disabilities or special needs.
Будьте осторожны
Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →