Software Quality Engineer (AI)

TL;DR

Software Quality Engineer (AI): Defining and implementing quality assurance strategies for enterprise-grade LLMs and AI agents with an accent on reliability and performance. Focus on designing scalable automation frameworks, integrating automated checks into CI/CD pipelines, and solving the unique challenges of generative AI output validation.

Location: Hybrid based out of London, San Francisco, Seattle, or New York City hubs

Company

hirify.global is an enterprise AI platform providing generative AI orchestration and LLMs for leading global companies to unlock enterprise-wide transformation.

What you will do

• Define and implement comprehensive QA strategies and test plans for AI agents and LLM-powered applications.
• Design and develop robust, scalable automation frameworks using TypeScript or Python.
• Collaborate with product managers, ML engineers, and data scientists to translate complex AI behaviors into validation criteria.
• Integrate automated checks within CI/CD pipelines to ensure rapid, high-quality releases.
• Perform root cause analysis on software defects and monitor production AI model performance.
• Design and maintain test data strategies and mock services for stable and repeatable test execution.

Requirements

• 5+ years of experience in software QA, focusing on complex distributed systems or AI/ML applications.
• Proficiency in Python or TypeScript and experience with modern frameworks like Playwright.
• Solid understanding of AI/ML concepts, model evaluation metrics, and generative AI testing challenges.
• Experience with cloud platforms (AWS, Azure, GCP) and containerization (Docker, Kubernetes) in CI/CD environments.
• Ability to drive initiatives independently in a fast-paced startup environment.
• Must be based in or able to work from the London, San Francisco, Seattle, or New York City hubs.

Culture & Benefits

• Generous PTO, company holidays, and 16 weeks of paid parental leave.
• Comprehensive medical and dental insurance, including early-detection cancer testing.
• Competitive pension scheme, company stock options, and annual wellness stipends.
• Learning and development stipend to support professional growth.
• Company-wide and team off-sites.