Operations Analyst, Safety (AI)

TL;DR

Operations Analyst, Safety (AI): Building and maintaining safety policies and tooling to protect AI-powered products with an accent on content moderation and abuse detection. Focus on triaging safety escalations, refining enforcement programs, and automating operational workflows to mitigate LLM-specific risks.

Location: Based in San Francisco, California. Relocation support is provided.

Salary: $190,000 - $300,000

Company

Advancing collaborative general intelligence to empower humanity through accessible AI tools and knowledge.

What you will do

• Review flagged content, safety escalations, and account-level abuse signals daily as a standing responsibility.
• Design and refine safety policies across the product stack in collaboration with engineering, legal, and security stakeholders.
• Build and maintain automation tooling, including triage agents and abuse detection frameworks, to accelerate casework.
• Partner with product teams to embed safety protections, such as model refusals and content flagging, into the user experience.
• Improve observability and detection for model safety trends and malicious behavior in production.

Requirements

• Must be based in or be able to relocate to San Francisco, California.
• 2+ years in operational trust & safety, content moderation, or fraud/abuse ops with direct case queue responsibility.
• Experience owning policy definition, operationalization, and enforcement end-to-end.
• Direct case experience with cybersecurity abuse, CBRN-relevant risk, youth safety, or prompt injection in production.
• Working familiarity with model safety and abuse risks like jailbreaks and scaled abuse.
• Practical experience using AI tools (e.g., Claude, Codex) to build or accelerate operational workflows.

Nice to have

• Experience with safety and integrity operations specifically for AI-powered products or LLM APIs.
• Track record of converting recurring case patterns into reusable tooling or process improvements.
• Experience training and onboarding other moderators to scale team output.

Culture & Benefits

• Generous health, dental, and vision benefits.
• Unlimited PTO and paid parental leave.
• Relocation support provided as needed.
• Visa sponsorship available for qualified candidates.