Research Scientist (Frontier Risk Evaluations)

TL;DR

Research Scientist (Frontier Risk Evaluations): Designing and creating evaluation measures, harnesses, and datasets to measure risks in frontier AI systems with an accent on agent robustness, AI control protocols, and high-risk capability testing. Focus on building ML pipelines, developing prototypes from research literature, and collaborating with government agencies to mitigate AI risks.

Location: Must be based in San Francisco, CA or New York, NY

Salary: $216,000 - $270,000 USD

Company

hirify.global is a leading data and evaluation partner for frontier AI companies, focused on bridging the gap between AI research and global policymakers to ensure safe and trustworthy AI deployments.

What you will do

• Design and build harnesses to test AI systems for dangerous capabilities, such as security vulnerability exploitation and CBRN uplift.
• Collaborate with government agencies and research labs to scope and design evaluations for advanced AI risks.
• Develop evaluation methodologies and author technical reports for policymakers.
• Turn research ideas into working prototypes and instrument ML pipelines.
• Publish findings and research to help the public and industry understand AI capabilities.

Requirements

• Location: Must be based in San Francisco, CA or New York, NY.
• At least 3 years of experience addressing sophisticated ML problems in research or product development.
• Proven track record of published research in machine learning, specifically in generative AI.
• Practical experience building ML pipelines and writing evaluation harnesses.
• Strong written and verbal communication skills for cross-functional operation.

Nice to have

• Experience crafting evaluations and benchmarks for LLM technologies.
• Experience with red-teaming or adversarial testing of AI systems.
• Familiarity with AI safety policy frameworks such as NIST AI RMF or the EU AI Act.

Culture & Benefits

• Comprehensive health, dental, and vision coverage.
• Equity-based compensation and retirement benefits.
• Learning and development stipend.
• Generous PTO and commuter stipend.
• Inclusive and equal opportunity workplace.

Hiring process

• Assessments focused on practical ML prototyping and debugging.
• Evaluation of research concepts and alignment with organizational culture.
• No LeetCode-style questions are used in the process.