AI Quality Assurance Analyst (AI)

TL;DR

AI Quality Assurance Analyst (AI): Ensuring the reliability and accuracy of AI-powered applications by evaluating model outputs and functional behavior with an accent on identifying hallucinations and edge cases. Focus on partnering with engineering teams to define quality standards for LLM-based products and maintaining robust test documentation.

Location: Must be based in the United States

Salary: $100,000–$110,000 per year

Company

hirify.global is a technology-driven company providing innovative solutions across employee benefits, retirement, wealth, and P&C business lines.

What you will do

• Design and execute comprehensive test plans for AI-powered applications.
• Evaluate AI responses for accuracy, consistency, and potential hallucinations.
• Test application workflows, UI behavior, and system integrations.
• Document and track defects using standard issue tracking tools like Jira.
• Collaborate with product managers and engineers to define acceptance criteria for AI features.
• Conduct regression testing to ensure stability as applications evolve.

Requirements

• Must be based in the United States
• Experience in a QA or testing role within a software or SaaS environment.
• Strong ability to evaluate AI-generated outputs for quality and appropriateness.
• Familiarity with prompt-driven systems and LLM failure modes.
• Proficiency in writing test cases and documenting technical defects.
• Understanding of AI governance and data privacy considerations.

Nice to have

• Prior experience testing AI or LLM-based products.
• Experience working in no-code or low-code environments.
• Familiarity with Linear or similar modern issue tracking tools.

Culture & Benefits

• Comprehensive health, wellbeing, and retirement benefits.
• Robust learning and development programs.
• Paid time off and financial benefits.
• Commitment to diversity, equity, and inclusion.
• Supportive culture focused on innovation and employee growth.