QA Engineer (Playwright, Python, AI)

TL;DR

QA Engineer (Playwright/Python): Architecting and evolving scalable automation frameworks for web, API, and AI-driven systems with an accent on maintainability and production-grade quality. Focus on designing Golden Dataset-based evaluation suites, hallucination detection, and integrating test suites into CI/CD pipelines.

Location: Remote, Hybrid, or Onsite (Sofia, Plovdiv, Stara Zagora, Nis). Requires an ET timezone work schedule

Company

hirify.global designs and scales complex software systems for global organizations in aerospace, automotive, telecom, and financial services.

What you will do

• Architect and evolve scalable Playwright (Python) automation frameworks for web, API, and end-to-end testing.
• Define automation strategies and integrate test suites into CI/CD pipelines with robust reporting and failure analytics.
• Validate AI-driven systems, including RAG pipelines, LLM integrations, and conversational workflows.
• Design and maintain Golden Dataset-based evaluation suites to measure grounding, faithfulness, and hallucination detection.
• Collaborate with ML engineers and backend teams to define evaluation criteria and quality thresholds for AI reliability.

Requirements

• Strong experience with Python test automation (e.g., Pytest, Unittest, Behave).
• Proven track record of building automation frameworks from scratch.
• Solid understanding of QA methodologies, test planning, and execution.
• Hands-on experience with smoke testing, regression testing, and CI/CD integration.
• Must be able to work according to an ET timezone schedule.
• Excellent communication skills and ability to work independently with minimal documentation.

Nice to have

• Experience in clinical, healthcare, or other regulated environments.
• Knowledge of PHP or React systems to support integration testing.
• Familiarity with CI/CD tools like Jenkins, GitHub Actions, or GitLab CI.

Culture & Benefits

• Supportive environment with expert mentorship and coaching.
• Personalized individual development plans tailored to career aspirations.
• Flexible work models: choose from remote, hybrid, or onsite options.
• Practical perks including food vouchers, additional health insurance, and sports cards.
• Engaging social events and a strong team spirit.