Research Scientist (AI Agent Robustness)

TL;DR

Research Scientist (AI Agent Robustness): Developing methods to ensure AI agents are safe, aligned, and robust against failures with an accent on benchmarking capabilities, adversarial testing, and risk mitigation. Focus on designing evaluation harnesses, building exploits for failure modes in tool-using agents, and characterizing multi-agent system risks.

Location: Must be based in San Francisco, CA; New York, NY; or Seattle

Salary: $216,000 - $270,000 USD

Company

hirify.global is a leading data and evaluation partner for frontier AI companies, providing full-stack technologies to help enterprises and governments build and oversee reliable AI applications.

What you will do

• Research AI agent capabilities and their relationship to safety, risk factors, and benchmarking methodologies.
• Design and build harnesses to test AI agents' tendency to take harmful actions under pressure or environmental trickery.
• Develop exploits and mitigations for failure modes arising from agent affordances like coding, web browsing, and computer use.
• Characterize and mitigate risks associated with systems involving multiple interacting AI agents.
• Collaborate with industry, public sector, and academia to publish research findings.

Requirements

• Must be based in the US (San Francisco, New York, or Seattle).
• At least 3 years of experience addressing sophisticated ML problems in research or product development.
• Practical experience in technical research, including building agent scaffolding and evaluation harnesses.
• Experience with post-training and RL techniques such as RLHF, DPO, and GRPO.
• Proven track record of published research in machine learning and generative AI.
• Strong written and verbal communication skills for cross-functional collaboration.

Nice to have

• Hands-on experience with agent evaluation frameworks like SWE-bench, WebArena, OSWorld, or Inspect.
• Experience with red-teaming, prompt injection, or adversarial testing of AI systems.

Culture & Benefits

• Comprehensive health, dental, and vision coverage.
• Retirement benefits and a dedicated learning and development stipend.
• Equity-based compensation for eligible roles.
• Generous PTO and commuter stipends.
• Inclusive and equal opportunity workplace culture.