Senior QA Automation Engineer (AI)

TL;DR

Senior QA Automation Engineer (AI): Designing and maintaining automated test suites and evaluation scenarios for AI-powered applications with an accent on scalable testing frameworks and AI/ML model validation. Focus on automating QA workflows using Python, building quality gates for CI/CD pipelines, and translating qualitative AI outputs into measurable metrics.

Location: London, UK

Company

hirify.global provides software tools that help innovators design and make a better world across architecture, engineering, construction, manufacturing, and media and entertainment.

What you will do

• Design and maintain automated test suites, evaluation scenarios, metrics, and test datasets for AI-powered features.
• Automate QA workflows using Python and CI/CD tools such as GitHub Actions and Jenkins.
• Create test harnesses and perform API testing for AI services and backend systems.
• Define quality strategies and acceptance criteria for functional, API, integration, and AI/ML features.
• Evaluate AI/ML model outputs for correctness and consistency using structured rubrics.
• Mentor teams on QA automation best practices and contribute to multi-team codebases.

Requirements

• Bachelor’s degree in Computer Science, Engineering, or equivalent experience.
• 5+ years of professional experience in software quality assurance or test automation.
• Strong programming skills in Python.
• Hands-on experience using AI-powered testing tools or AI coding assistants.
• Proficiency in CI/CD tools and designing QA frameworks used by multiple teams.
• Must be based in London, UK

Nice to have

• Experience testing AI/ML applications or validating machine learning models.
• Familiarity with ML frameworks such as PyTorch and TensorFlow.
• Experience with data validation tooling (e.g., Great Expectations) or orchestration tools (e.g., Airflow, Metaflow).
• Knowledge of MLOps practices, including model monitoring and drift detection.
• Experience with cloud providers (AWS, Azure, GCP) and experiment tracking tools (MLflow, Comet, Weights & Biases).

Culture & Benefits

• Culture of belonging designed to help every employee thrive.
• Competitive compensation package including base salaries and annual cash bonuses.
• Stock grants and comprehensive benefits package.
• Opportunity to work on high-impact software used in global industries.
• Collaborative environment with teams across various global locations.