Senior QA Engineer (AI)

TL;DR

Senior QA Engineer (AI): Ensuring quality across complex data flows and generative AI products with an accent on conversational agents and LLM validation. Focus on designing non-deterministic assertions, detecting hallucinations and bias, and validating structured data outputs in AI-driven systems.

Location: Must be based in the UAE (Dubai or Abu Dhabi)

Company

hirify.global is a company focused on developing complex data flows and AI-driven conversational products.

What you will do

• Lead QA efforts for data pipelines, APIs, and conversational AI agents.
• Design and execute generative AI testing scenarios, focusing on intent recognition and edge-case behavior.
• Implement LLM-specific validations, including prompt grounding and golden set assertions.
• Conduct critical safety audits to detect hallucinations, bias, toxicity, and PII leakage.
• Establish real-time testing approaches for streaming data and API monitoring.
• Collaborate with ML and NLP teams to define comprehensive evaluation metrics for agent performance.

Requirements

• 5+ years of experience in QA engineering, specifically with data systems or AI applications.
• Proficiency in Python and SQL for automation scripting and data validation.
• Strong experience in API testing using REST and GraphQL.
• Familiarity with LLMs, chatbot frameworks, and conversational testing methodologies.
• Must be based in the UAE (Dubai or Abu Dhabi).

Nice to have

• Experience integrating LLM-based assistants like GitHub Copilot or ChatGPT into the QA lifecycle.
• Exposure to QA automation tools such as PyTest, Postman, or dbt tests.