Data Scientist, Foundation AI (AI Engineering)

TL;DR

Data Scientist, Foundation AI (AI Engineering): You will play a critical role in evaluation and optimization for user-facing GenAI systems (such as text, image, video, 3D, 4D). Focus on defining how we measure safety, responsibility, quality, and efficiency combining annotation analysis, design of experiments, causal inference, model-based evaluation methods (such as LLM-as-a-judge), optimization algorithm, and AI models to drive product decisions and model improvements.

Location: Must be based in San Mateo, CA

Salary: $185,860—$221,380 USD

Company

hirify.global is building the tools and platform that empower our community to bring any experience that they can imagine to life.

What you will do

• Develop Evaluation Frameworks: Design and operationalize rigorous evaluation systems for either GenAI features (text, image, video, 3D, 4D).
• Run Rigorous Experiments: Conduct online experiments (A/B tests) and causal inference to quantify the impact of GenAI features.
• Define Success Metrics: Partner with cross-functional teams to define leading/lagging indicators for GenAI feature user satisfaction, business success, and safety.
• Build Automated Systems: Research and apply state-of-the-art methodologies to build reproducible evaluation tooling that lift rigor and efficiency across the company.
• Conduct Applied Research at the Frontier: Maintain an active pulse on the intersection of Gen AI and Data Science.

Requirements

• Possess or pursuing a PhD or equivalent in Statistics, Economics, Computer Science, Applied Math, Physics, Engineering, or a related quantitative field.
• Technical Proficiency: Strong proficiency in SQL (Hive/Spark) for manipulating large datasets and scripting languages (Python or R) for analysis and modeling.
• Experimentation and Causal Inference: A solid grounding in experimentation, causal inference, and statistical analysis, including test design and metric design for feature impact.
• Problem Solving: A demonstrated track record of framing ambiguous problems, designing analytical approaches, and solving open-ended data science problems that drive business impact.
• Learning Agility: Ability to effectively and responsibly use AI tools to enhance productivity and a passion for continuously improving methods in a fast-evolving field.
• GenAI Familiarity: Familiarity with GenAI models and safety/quality evaluation methods.

Nice to have

• Applied Research Background: A track record of applied research or publications in relevant technical fields is highly valued.
• Expertise in the model training lifecycle is a plus (e.g., fine-tuning, RLHF, or synthetic data generation).

Culture & Benefits

• Roles that are based in an office are onsite Tuesday, Wednesday, and Thursday, with optional presence on Monday and Friday (unless otherwise noted).
• All full-time employees are also eligible for equity compensation and for benefits.