Product Data Scientist (AI)

TL;DR

Product Data Scientist (AI): Measuring the impact of AI agents and model changes on real-world performance with an accent on experimental design and metric definition. Focus on designing A/B tests, conducting root-cause analyses of model regressions, and building analytical frameworks to drive the ML roadmap.

Location: Hybrid in San Francisco

Company

hirify.global is an infrastructure platform for enterprises to build and orchestrate autonomous AI workforces.

What you will do

• Define and track product and agent-level metrics to measure real-world performance and automation rates.
• Design, run, and interpret A/B tests for model changes, prompts, and agent behavior.
• Analyze conversational and unstructured data to identify opportunities and explain performance differences.
• Build statistical models, simulations, and analytical frameworks to support product and ML decisions.
• Partner with Engineering to improve instrumentation, data quality, and analytical data models.
• Develop dashboards and self-serve tools to monitor product and agent performance.

Requirements

• 4+ years of experience in Data Science, Product Analytics, or a highly quantitative product role.
• Advanced proficiency in SQL and Python.
• Strong expertise in experimental design, A/B testing, statistics, and causal inference.
• Ability to translate ambiguous product questions into rigorous analyses.
• Experience partnering closely with Product, Engineering, or Machine Learning teams.
• Must be based in San Francisco for hybrid work.

Nice to have

• Experience with LLMs, AI agents, or generative AI production impact measurement.
• Familiarity with ML evaluation systems and NLP/speech data analysis.
• Experience in enterprise B2B products.
• Background in fast-growing startups or ambiguous environments.

Culture & Benefits

• High-growth AI startup backed by a16z and Y Combinator.
• Competitive salary and equity package.
• Comprehensive healthcare, dental, and vision coverage.
• Culture of extreme ownership, craftsmanship, and meritocracy.
• Collaborative, friendly, and inclusive team environment.