AI Agentic Workflow Reviewer (AI)

Location: In office in San Jose, California, United States

Expected hourly salary: $50–$55 per hour, based on experience, skills, and qualifications.

Company

hirify.global is a global data engineering company providing data, evaluation frameworks, human expertise, and solutions for Generative AI and AI systems.

What you will do

• Review multi-step agent task trajectories against rubrics covering task success, safety, and policy adherence.
• Assess whether AI agents respect user agency, informed consent, and the intent behind instructions.
• Apply privacy guardrails when reviewing tasks involving sensitive or personal data.
• Work in isolated test environments and verify environment resets between test runs.
• Flag and escalate safety-relevant or ambiguous findings through defined escalation paths.

Requirements

• Bachelor’s degree or equivalent practical experience.
• 2–5+ years of experience in quality review, QA/QC, trust and safety, content moderation, data annotation, or a similar judgment-intensive review role.
• Strong written communication and the ability to clearly justify scoring decisions for audit.
• Comfort with ambiguity and structured decision-making under a rubric.
• Baseline understanding of AI/ML concepts and how AI agents complete tasks.

Culture & Benefits

• Work in a dedicated reviewer team evaluating complex, real-world agentic AI workflows.
• Use structured rubrics and isolated environments to help define safe and policy-adherent agent behavior.
• Role is performed in an office setting in San Jose, California.