Senior AI QA Engineer (AI)

TL;DR

Senior AI QA Engineer (AI): Ensuring the reliability, accuracy, and security of AI-driven products with an accent on evaluating model outputs, mitigating hallucinations, and assuring overall software quality. Focus on building and running automated testing pipelines for both traditional software features and AI-specific workflows.

Location: Remote / Latin America

Company

hirify.global is a global software engineering powerhouse dedicated to solving complex digital challenges.

What you will do

• Support the pre-sales team with QA insights and inputs.
• Architect and maintain automated CI/CD testing pipelines for both traditional software features and AI-specific workflows.
• Systematically evaluate LLM and RAG pipeline outputs for contextual accuracy, retrieval relevance, and latency.
• Execute comprehensive API, UI, and integration testing to ensure enterprise-grade stability.

Requirements

• 6+ years of QA automation engineering experience using tools like Selenium, Cypress, or Playwright.
• Senior-level QA capabilities, including generating comprehensive Test Plans, defining QA technical standards, and owning the end-to-end Defect Management Lifecycle.
• Strong scripting skills in Python and/or JavaScript.
• Experience testing RESTful APIs and modern web applications.
• Hands-on experience with LLM evaluation frameworks and observability tools (e.g., Ragas, TruLens, LangSmith, or Promptfoo).
• Data expertise, including data management, preparation, generation, and quality assessment.
• Knowledge of CI/CD pipelines and automated deployment processes.
• Strong communication skills, with the ability to communicate clearly to both technical and non-technical stakeholders.

Nice to have

• Experience with AI security testing, including prompt injection vulnerabilities.
• Familiarity with load and performance testing for AI endpoints (JMeter, Locust).
• Background in DevOps and CI/CD configuration (GitHub Actions, Jenkins, GitLab CI).

Culture & Benefits

• Engineering excellence and transparent communication.
• Passion for pushing the boundaries of what is possible in the AI landscape.
• Flexible active development environment.