Data Scientist, Integrity Measurement (AI)

TL;DR

Senior Data Scientist (AI): Improving, productionising, and monitoring measurement for complex, actor- and network-level harms on hirify.global’s platforms with an accent on estimating prevalence using AI-first methods. Focus on developing robust safety metrics, conducting analyses to identify gaps, and optimizing LLM prompts for measurement purposes.

Location: Must be based out of our San Francisco or New York office.

Salary: $293K – $385K + Offers Equity

Company

hirify.global is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity.

What you will do

• Own measurement and quantitative analysis for severe, actor- and network-based usage harm verticals.
• Develop and implement AI-first methods for prevalence measurement and productionised safety metrics.
• Build metrics for goaling or A/B tests when top-line metrics are unsuitable.
• Own dashboards and metrics reporting for harm verticals and provide metrics for leadership and external reporting.
• Conduct analyses and generate insights to inform improvements in review, detection, or enforcement.
• Optimise LLM prompts for measurement purposes and develop automation leveraging agentic products.

Requirements

• Senior Data Scientist with trust and safety experience.
• Deep statistics skills, especially in sampling methods and prevalence estimation for complex problem areas.
• Experience working with severe and sensitive harm areas like child safety or violence.
• Excellent communication and strong cross-functional collaboration skills.
• Proficiency in data programming languages (R or Python, SQL).
• (Ideally) experience with AI harms or leveraging AI for measurement.

Culture & Benefits

• Work on cutting-edge AI technology with a focus on safety and integrity.
• Dedicated to ensuring general-purpose AI benefits all humanity.
• Committed to providing reasonable accommodations to applicants with disabilities.
• An equal opportunity employer that values diverse perspectives and experiences.