AI Agent Evaluator

Location: Remote, United States; candidates must be based in the United States

Salary: $25–$40 per hour

Company

hirify.global provides consulting services for technology and AI-related projects.

What you will do

• Test two AI agents side by side on everyday tasks, including shopping, travel, bookings, bill payments, email follow-ups, and forms.
• Approve or stop agents at key steps during task execution.
• Screen-record each testing session and collect supporting evidence.
• Score task completion, quality, control issues, user satisfaction, and time.
• Provide clear written feedback on agent performance and usability.

Requirements

• Must be based in the United States and hold a bachelor's degree.
• 2–3 years of experience, preferably in engineering, business management, or office operations.
• Regular use of AI tools in daily life.
• Strong attention to detail and ability to follow instructions.
• Excellent written communication and documentation skills.
• An active personal AI account on the Free, Go, or Plus plan, excluding Pro, and a smartphone capable of screen recording.

Nice to have

• Relevant domain expertise.
• No coding or AI development experience is required.

Culture & Benefits

• Remote project-based contractor engagement.
• Each testing run takes approximately 1.5 hours, including evidence collection and scoring.
• Kickoff is scheduled for Monday, October 12.
• Onboarding takes place before the project begins, subject to scheduling.

Hiring process

• Submit an application and complete screening questions.
• Attend a 15–30-minute interview.
• Complete profile review and onboarding.