7 дней назад
Mid-Level Generative AI and LLM QA Automation Engineer (AI)
Вакансия из Hirify Global, списка международных tech-компанийДля мэтча и отклика нужен Plus
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Mid-Level Generative AI and LLM QA Automation Engineer (AI/LLM): Building automated testing frameworks, evaluation datasets, and CI/CD validation pipelines for conversational AI and productivity tools with an accent on hallucination detection, output evaluation, and AI observability. Focus on designing adversarial dialogue tests, analyzing model behavior and bias, and integrating continuous quality and compliance checks.
Location: Porto, Portugal — hybrid, with up to 4 remote days per week. Candidates must have the legal right to work in Europe.
Company
is an international IT company delivering advanced AI solutions.
What you will do
- Develop and maintain automated test frameworks using Playwright, Cypress, Selenium, or similar tools.
- Write automation scripts in Python and JavaScript/TypeScript.
- Design evaluation datasets and automated evaluation pipelines, including LLM-as-judge methods.
- Analyze AI outputs for hallucinations, errors, bias, and model behavior issues.
- Integrate AI and functional tests into CI/CD pipelines for continuous validation.
- Create conversational AI test cases covering dialogue flows, edge cases, adversarial prompts, fairness, bias mitigation, and compliance.
Requirements
- 3+ years of experience in QA/SDET roles with hands-on automation development.
- Proficiency with Playwright, Cypress, Selenium, or similar automation tools.
- Strong scripting skills in Python and JavaScript/TypeScript.
- Knowledge of AI testing challenges, including prompt regression, hallucination detection, and output evaluation.
- Experience with evaluation datasets, AI observability tools, and CI/CD integration.
- Fluent English, both verbal and written, and the legal right to work in Europe.
Nice to have
- Experience with AI-assisted test generation and self-healing test scripts.
- Knowledge of LLM evaluation strategies.
- Relevant QA or AI safety certifications.
Culture & Benefits
- Hybrid work model in Porto with up to 4 remote days per week.
- Opportunity to work on conversational AI, AI Rater, and productivity tools.
- Work in a dynamic environment focused on AI quality, safety, performance, and compliance.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →