AI QA Engineer (LLM)

Job title: AI QA Engineer Type: Full-time, Contract running until September

Location: Anywhere in the EU | Flexible remote, hybrid, or office working

The role

We're looking for a QA Engineer to work across product quality, AI behaviour evaluation, and defect investigation.

This is not a traditional QA role focused purely on scripted testing or automation. A core part of the role involves working within the agentic software delivery lifecycle — identifying failure modes in AI agent outputs, patching prompts and pipelines to resolve them, validating those changes, and returning improved builds to production. Because AI behaviour is not always deterministic, strong judgement, attention to detail, and communication skills are essential.

You'll also review AI agent conversations and assess whether responses were accurate, appropriate, and aligned to the expected customer experience, supporting release testing and regression prevention across the platform. Outside release periods, you'll help improve automated coverage, refine QA processes, and investigate product or AI-related issues raised internally.

You'll investigate production and customer-reported issues, identifying whether problems are expected behaviour, configuration issues, or genuine defects before escalating clear issues to engineering teams.

We're a small team, so we need someone who can take genuine ownership of this function — setting things up, anticipating problems before they arise, and driving quality forward without heavy hand-holding.

Required experience

• 5+ years of QA or software delivery experience
• Hands-on experience with the agentic software delivery lifecycle: identifying failure modes, patching prompts and pipelines, and validating fixes through to production
• Experience with agentic tracing and observability tools such as LangFuse, LangGraph, or similar
• Strong manual QA experience
• Experience building or maintaining automated test coverage
• Experience working with AI, LLM, or agentic products as part of your day-to-day workflow
• Strong written English and communication skills
• The ability to assess nuanced conversations and provide clear feedback
• Confidence investigating issues independently and making sound judgement calls

Desirable experience

• Broader experience with AI tooling, agent frameworks, or conversational AI platforms
• Experience in a setup or greenfield quality function, where you've had to define processes rather than inherit them

What you'll get

• A varied role combining QA, AI evaluation, automation improvement, and product investigation rather than repetitive manual testing
• The opportunity to work on AI-driven products where quality depends on judgement and context, not just predefined test cases
• Real ownership over how quality processes evolve as AI capabilities continue to grow
• Exposure to modern AI and agentic systems in a production environment
• The chance to work closely with Product, Engineering, and AI teams on complex problems where there often isn't a single correct answer

About hedgehog lab

We're a digital product consultancy with 20 years of experience, helping organisations solve complex problems through technology, design, and delivery.

Ready to apply?

Submit your application below.

Please note: we're unable to offer visa sponsorship for this role.

Показать контакты