Senior QA Automation Tester (GenAI)

TL;DR

Senior QA Automation Tester (GenAI): Ensuring quality, reliability, and correctness of web applications, mobile, and GenAI-enabled systems with an accent on UI workflows, NLP/GenAI output validation, and defect tracking. Focus on designing test cases for non-deterministic LLM outputs, validating functional correctness, format adherence, edge cases, and identifying hallucinations or unsafe responses.

Location: Onsite - Hybrid in Evanston, Illinois, USA (3 days a week in office)

Company

Digital Product Engineering company scaling globally with 17000+ experts across 39 countries and dynamic, non-hierarchical culture.

What you will do

• Test UI workflows using Selenium, Cypress, or Playwright, validating cross-browser behavior, responsiveness, and user interactions.
• Validate GenAI/NLP features including chat interfaces, text generation, summarization, and extraction outputs for correctness, schema adherence, and edge cases.
• Design test cases for non-deterministic LLM outputs, prompt variations, response consistency, hallucinations, and unsafe responses.
• Document defects with reproducible steps, maintain test plans, cases, QA reports, and track coverage metrics.
• Support QA governance through structured documentation and reporting across releases.

Requirements

• Minimum 7 years of experience as QA Automation
• AI testing exposure, API testing, Playwright with Java/Python (strong)
• Experience testing web applications and mobile
• Hands-on with Postman for API validation and Selenium for UI automation
• Strong analytical skills for edge cases, failure scenarios, clear test documentation, and defect reports

Nice to have

• Experience testing NLP or GenAI-based applications and validating LLM outputs/prompts
• CI/CD-integrated testing, performance/load testing tools
• Basic database validation for test verification