Senior Test Automation Engineer (AI/ML)

Location: Madrid, Spain

Company

hirify.global is a wealth management bank developing AI-driven capabilities and an ML/MLOps platform for business-critical use cases.

What you will do

• Define and evolve the technical test automation approach and framework architecture for the ML & AI ART.
• Design reusable automation patterns for UI, API, service, data pipeline, contract, and BDD testing.
• Develop AI/ML-specific evaluations for LLM outputs, RAG systems, agentic workflows, and ML pipelines.
• Integrate, orchestrate, and schedule automated tests across development, integration, UAT, and pre-production environments using GitLab CI/CD.
• Manage flaky tests, investigate failures, raise defects with evidence, and maintain audit-ready traceability from Jira stories to test results.
• Collaborate with Product Owners, Scrum Masters, ML Engineers, MLOps Engineers, Data Engineers, Architects, and Test Managers across squads.

Requirements

• 5–7 years of test automation experience with substantial hands-on Python experience and ownership of automation framework design.
• Practical expertise with Playwright-Python, requests, pytest, Behave or pytest-bdd, and Gherkin.
• Experience with CI/CD pipelines, distributed and parallel test execution, flaky-test management, reporting tools, Git, Docker, and Kubernetes basics.
• Understanding of APIs, microservices, event-driven architectures, authentication, information security, data privacy, and compliant test data handling.
• Experience with Jira and test management integrations such as Xray or Agile Hive, plus knowledge of SAFe and DevOps principles.
• Fluent English is required; German is a plus. A Bachelor's or Master's degree in Computer Science, Engineering, or a related field is expected.

Nice to have

• Experience testing AI/ML systems, including LLM evaluations, RAG, and agentic workflows.
• Experience in regulated financial services, healthcare, or pharmaceutical environments.
• ISTQB Foundation certification and SAFe or DevOps certifications.

Culture & Benefits

• Work within an ML & AI Agile Release Train and contribute to PI Planning, System Demos, and Inspect & Adapt ceremonies.
• Operate within the Bank's Test Strategy, Test Policy, reporting requirements, and audit governance.
• Use synthetic or anonymised test data wherever possible to support confidentiality expectations.
• Collaborate across multiple engineering and product disciplines with a strong focus on ownership, quality, and reliable delivery.