Назад
Company hidden
14 дней назад

Lead QA Engineer (AI Voice)

Формат работы
remote (только Poland)/onsite
Тип работы
fulltime
Грейд
lead
Английский
b2
Страна
Poland
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Lead QA Engineer (AI Voice) (Voice AI, LLM, Python/JavaScript): Designing and leading quality assurance for voice AI systems covering speech input, ASR, LLM reasoning, tools, integrations, and speech output with an accent on automation, multilingual coverage, and measurable voice-quality evaluation. Focus on testing non-deterministic AI behaviour, leading UAT, managing release risks, and validating latency, interruptions, accents, noise, intent accuracy, and task completion.

Location: Poland; remote work available, with office work as an option

Company

hirify.global is an established technology services company operating since 2008 with more than 1,800 talents across seven global sites.

What you will do

  • Design the voice AI testing strategy and process across speech input, ASR, LLMs, tools, integrations, and speech output.
  • Define a scalable automation approach with multilingual coverage of approximately 100 languages.
  • Test predictable flows such as intents, routing, and integrations alongside non-deterministic AI behaviour.
  • Evaluate prompts and LLM responses using test datasets, scoring criteria, and voice-quality metrics.
  • Lead UAT by planning scenarios, preparing test data, coordinating business testers, and recommending sign-off.
  • Own defect and risk management, release go/no-go recommendations, stakeholder communication, and mentoring of the Regular engineer.

Requirements

  • At least 6 years of software QA experience, including at least 2 years testing voice agents or conversational AI in live delivery.
  • Experience leading E2E and UAT activities and working as the lone or lead QA with limited guidance.
  • Experience working in international or distributed teams.
  • Practical voice AI testing experience covering speech recognition and output, latency, interruptions, accents, and noise.
  • Knowledge of voice-quality metrics including word error rate, intent accuracy, and task completion.
  • Practical understanding of how LLMs work and fail; deep LLM engineering is not required.
  • Test automation with Python or JavaScript/TypeScript, CI/CD, and API testing such as Postman.
  • Experience with cloud speech services such as Azure, Google, AWS, or similar for generating test calls.
  • Experience with Jira or Azure DevOps, a test management tool, and troubleshooting through logs, transcripts, and call recordings.
  • Structured, recommendation-first communication with technical and business stakeholders.
  • English proficiency of at least B2.

Nice to have

  • Experience with voice testing platforms such as Hamming, Cekura, Coval, or Botium.
  • Experience with LLM evaluation tools such as Promptfoo, DeepEval, or LangSmith.
  • Telephony knowledge including SIP, Twilio, or Azure Communication Services.
  • ISTQB certification.
  • Responsible-AI testing experience covering PII, bias, and safety.
  • Experience validating customer journeys across multiple markets.

Culture & Benefits

  • Stable employment with a company operating since 2008 and a diverse, inclusive, values-driven community.
  • Remote or office work under an “Office as an option” model, with flexibility regarding working hours and the preferred contract form.
  • Workation available in line with company policy.
  • Online onboarding with a Buddy, cooperation with experienced engineers, and access to learning platforms and certificate training.
  • Upskilling programs, Competency Centers, knowledge-sharing sessions, community webinars, and more than 110 training opportunities annually.
  • Health and well-being activities, modern office equipment, and opportunities to support charities and the environment.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →