Research Engineer (AI)

TL;DR

Research Engineer (AI): Developing and implementing novel safety methods to protect AI systems from misuse and malicious activity with an accent on misuse detection, model safeguards, and adversarial robustness. Focus on building classifiers for abuse patterns, creating evaluation methodologies for agentic settings, and transferring research prototypes into production defenses.

Location: Hybrid: Must be based in San Francisco, CA or New York City, NY (office attendance at least 25% of the time)

Salary: $350,000 - $850,000 USD per year

Company

Public benefit corporation dedicated to creating reliable, interpretable, and steerable AI systems.

What you will do

• Lead research projects to detect Claude misuse and identify malicious organizations and accounts.
• Design and execute offline analyses of model usage data to build classifiers and detection systems.
• Develop prototypes for real-time safeguard signals and partner with engineers for tech transfer.
• Research methods for detecting abusive behavior in chat-based and agentive workflows.
• Create evaluations and methodologies to measure safeguard effectiveness in agentic settings.
• Communicate findings to Trust & Safety, research, and product teams to inform strategic decisions.

Requirements

• Track record of independently driving technical research projects in AI, ML, security, or integrity.
• Proficiency in Python and experience working with large datasets.
• Working familiarity with LLM operations, including sampling, prompting, and training.
• Bachelor’s degree or equivalent professional experience in a relevant technical field.
• Must be based in or be able to work from the San Francisco or New York City offices.

Nice to have

• Experience training ML models for abuse, fraud, or security applications.
• Expertise in LLM evaluation methodologies and design.
• Background in Trust & Safety, threat intelligence, or adversarial ML.
• Experience with red teaming, jailbreaking, or interpretability methods like steering vectors.
• History of transferring research prototypes into production systems.

Culture & Benefits

• Collaborative "big science" environment focused on high-impact, large-scale research.
• Competitive compensation and optional equity donation matching.
• Generous vacation and parental leave policies.
• Flexible working hours and modern office spaces.
• Visa sponsorship availability for qualified candidates.