Senior QA Automation Engineer (AI Voice)

Location: Remote, candidates accepted only from Latin America; reliable overlap with US Eastern business hours required.

Company

IDT Technologies is part of hirify.global, an American telecommunications company providing prepaid communications, payment services, international voice, and unified cloud communications solutions.

What you will do

• Own the end-to-end test strategy for an AI voice virtual agent across LLM, speech, and telephony layers.
• Design complex conversational scenarios covering multi-turn dialogues, interruptions, ambiguous input, escalation, recovery, prompt injection, hallucinations, and data disclosure risks.
• Build and maintain evaluation frameworks for non-deterministic outputs using golden datasets, semantic or rubric-based assertions, and automated regression detection.
• Provision SIP endpoints and softphones, automate real call origination, control media conditions, and troubleshoot call flows.
• Extend automated test coverage with Python, Playwright, JsSIP, and WebRTC, integrating suites into Jenkins and GitHub Actions as release gates.
• Define quality metrics, maintain traceability in Allure TestOps, and communicate evidence-based quality risks to stakeholders.

Requirements

• 5+ years of QA experience with recent hands-on ownership of a complex integrated product.
• Practical experience testing SIP/VoIP systems, including call flows, signaling, media handling, softphones, SIP trunking, SIP traces, or packet captures.
• Professional fluency in English, spoken and written, with reliable overlap with US Eastern business hours.
• Proficiency in Python and experience building and maintaining automation frameworks.
• Experience testing non-deterministic or probabilistic systems and defining analytical quality metrics for release decisions.
• Hands-on experience with formal TMS tools and Jira.

Nice to have

• Experience testing LLM-based or conversational AI products, evaluation frameworks, or LLM-as-a-judge approaches.
• Knowledge of ASR, transcription accuracy, WER, TTS, endpointing, and silence detection.
• Experience with RAG, retrieval-grounded systems, WebRTC media testing, codecs, jitter, or packet loss.
• Background in IVR, contact center, or conversational IVR platforms.
• Multilingual testing, especially Spanish/English code-switching, and prompt engineering literacy.

Culture & Benefits

• Remote B2B engagement for candidates in Latin America.
• Stable long-term role with annual performance reviews and career growth opportunities.
• Hardware support and compensation for professional training, seminars, and conferences.
• Company-supported English classes and a referral program.
• Work with an international telecommunications and cloud communications organization.