Senior QA Engineer (AI)

TL;DR

Senior QA Engineer (AI): Testing and optimizing AI-powered assistants and agents across mobile and TV platforms with an accent on prompt following, AI observability, and voice interaction. Focus on analyzing agent traces via LangSmith, verifying distributed backend integrations, and ensuring high-quality end-to-end product scenarios.

Location: Fully remote across Europe

Company

hirify.global develops award-winning AI products, focusing on speech technologies, NLP, and Generative AI for corporate clients.

What you will do

• Assess AI assistant quality, specifically prompt/instruction following and regression testing between versions.
• Analyze agent behavior "under the hood" using observability tools like LangSmith to verify tool calls and memory handling.
• Perform E2E manual testing on mobile and TV devices, focusing on voice UX and ASR/TTS quality.
• Test distributed backend services, including dialog orchestration and LLM agents, verifying request paths and service seams.
• Maintain test cases and bug reports while collaborating closely with TPMs, Prompt Engineers, and ML developers.

Requirements

• 5+ years of manual QA experience with a high level of ownership and self-direction.
• Experience testing AI agents, specifically checking response quality and prompt following.
• Proficiency with AI observability and tracing tools (e.g., LangSmith).
• Experience with API testing (REST/gRPC) and analyzing logs/traces (Kibana/Grafana/Loki).
• Experience testing mobile and/or TV applications.
• Must be based in Europe.

Nice to have

• Experience testing speech technologies (ASR/TTS) and working with audio.
• Knowledge of AI evaluation methods (LLM-as-judge, scenario-based self-play).
• CI/CD experience (GitHub/GitLab) and hands-on experience with ADB.
• Willingness to transition into test automation over time.

Culture & Benefits

• Work with a cutting-edge stack: Speech Tech, NLP, Generative AI, and voice-first agentic architecture.
• High engineering bar with a focus on production-ready quality and real ownership.
• Startup pace with enterprise stability and direct impact on real-world products.
• Fully remote work environment across Europe.
• 21 vacation days, public holidays, and 5 sick days.
• Private English lessons via Preply.