AI QA Engineer (AI)

TL;DR

AI QA Engineer (AI): Design, implement, and maintain automated tests for systems using AI/ML models with an accent on integrating tests into CI/CD pipelines and monitoring regression runs. Focus on creating comprehensive test plans covering AI-specific quality aspects, assessing datasets for quality and bias, and applying machine learning knowledge to interpret AI results.

Location: Primarily based in Linz, AT with a hybrid work model offering up to 50% home office and a Work From Abroad Program (up to 40 days).

Salary: €3,843 gross per month (minimum, with market-competitive overpayment possible).

Company

hirify.global is a leader in remote connectivity software, harnessing cutting-edge technologies such as Augmented Reality and Artificial Intelligence to drive digitalization.

What you will do

• Design, implement, and maintain automated tests, especially for systems using AI/ML models.
• Integrate tests into CI/CD pipelines, monitor regression runs, and adjust strategies for AI components.
• Create and maintain comprehensive test plans covering AI-specific quality aspects (performance, accuracy, robustness, fairness).
• Contribute to scrum ceremonies, providing feedback from both user and AI quality perspectives.
• Collaborate with Product Owners, Tech Leads, Developers, and AI Engineers to ensure testability from use case definition to model refinement.
• Assess datasets for quality and bias and evaluate their impact on model behavior.

Requirements

• Experience in test automation, software development, or quality assurance, with proficiency in Python or similar languages.
• Strong understanding of QA principles, SDLC, and foundational machine learning concepts.
• Ability to design meaningful AI/ML test scenarios, focusing on model behavior, edge cases, and reliability.
• Knowledge of evaluation metrics and basic statistical methods (precision, recall, F1, AUC, distributions, sampling).
• Experience assessing datasets, including labeling, data quality checks, and identifying bias, data drift, or concept drift.
• Familiarity with AI evaluation tools (e.g., MLflow, Weights & Biases) and standard automation or CI/CD tools (e.g., Selenium, GitLab, Jenkins).

Culture & Benefits

• 28 days of holidays.
• Onsite onboarding in HQ office.
• Compensation and benefits packages including company achievement bonus and stock-based options, with regular salary reviews.
• Hybrid and flexible work time with up to 50% home office.
• Work From Abroad Program allowing up to 40 days of work outside your contracting country.
• Access to Corporate Benefits platform with many discounts, fresh organic fruits, muesli, soft drinks and snacks.