Data Scientist (AI)

TL;DR

Data Scientist (AI): Exploring and analyzing large-scale datasets from AI evaluations to uncover causal relationships and model behaviors with an accent on statistical methodology and experimental design. Focus on building reproducible analysis pipelines and partnering with ML researchers to improve the reliability and interpretability of frontier AI systems.

Location: Onsite in the Bay Area, USA

Company

hirify.global is an open platform dedicated to evaluating AI model performance in real-world scenarios through community feedback and rigorous, transparent leaderboards.

What you will do

• Analyze large, complex datasets to identify patterns, biases, and causal relationships in AI model behavior.
• Formulate and test hypotheses regarding data quality and model performance through experimental design.
• Develop and maintain reproducible analysis pipelines using Python, Pandas, NumPy, and Spark.
• Design metrics and statistical frameworks that evaluate performance across diverse domains and tasks.
• Communicate data-driven insights to both technical and non-technical stakeholders to guide research directions.

Requirements

• 6+ years of experience in data science, ML analytics, or applied research.
• Strong proficiency in Python, Pandas, NumPy, and distributed computing frameworks like Spark.
• Expertise in statistical modeling, causal inference, and experimental design.
• Deep understanding of data distributions, sample quality, and data distribution shifts.
• Strong communication skills and ability to collaborate with cross-functional ML research teams.

Nice to have

• Background in AI model evaluation.
• Experience working with LLM outputs, embeddings, or large-scale model artifacts.
• Hands-on experience with A/B testing methodologies.

Culture & Benefits

• Competitive compensation packages including equity.
• Comprehensive health, dental, and vision insurance.
• Collaborative and high-energy work environment with experts from top-tier research institutions.
• Opportunity to contribute to a mission-driven, transparent, and impact-focused AI organization.