Senior QA Automation Engineer (AI)

TL;DR

Senior QA Automation Engineer (AI): Designing and maintaining automated test suites for web applications and APIs with an accent on LLM quality evaluation and hallucination mitigation. Focus on building end-to-end UI automation with Playwright and developing custom Python scripts for AI-powered feature validation.

Location: Remote, Hungary

Company

hirify.global is a digital product engineering company that builds products, services, and experiences at scale for global clients across various industries.

What you will do

• Design, develop, and maintain automated test suites for web applications, APIs, and AI-powered features.
• Evaluate LLM outputs for accuracy, relevance, consistency, and hallucination risks.
• Create prompt tests and quality checks using Promptfoo or similar LLM evaluation frameworks.
• Build end-to-end UI automation using Playwright following page object models and maintainable architecture.
• Develop reusable Python scripts for API testing, automation utilities, and LLM quality evaluation.
• Collaborate with product managers and developers to define test strategies and communicate quality risks.

Requirements

• Must be based in Hungary.
• 5+ years of experience in QA Automation and software testing.
• Hands-on experience testing AI or LLM-based features, including prompt testing and output validation.
• Strong proficiency in Playwright for end-to-end UI automation.
• Solid Python scripting skills for automation and custom quality checks.
• Experience with Postman and REST API testing, including CI/CD integration.

Nice to have

• Experience with mobile test automation using Appium, XCUITest, or similar tools.
• Understanding of iOS and Android platform-specific testing considerations.
• Experience integrating automated tests into CI/CD pipelines.
• Familiarity with MCP or agent-assisted testing approaches in Playwright.

Culture & Benefits

• Dynamic and non-hierarchical work culture.
• Opportunity to work within a global network of 18,000+ experts across 39 countries.
• Focus on professional growth and solving complex challenges for high-profile global clients.