GenAI Quality Engineer (m/f/d)

Location: Sofia, Bulgaria; hybrid work model with 3 days per week in the Sofia office.

Company

Tieto Tech Consulting provides design-led, data-centric, and AI-powered digital engineering and consulting services to enterprises worldwide and is part of hirify.global.

What you will do

• Validate LLM-based applications and AI-generated outputs.
• Evaluate the quality, reliability, and behavior of probabilistic AI systems.
• Create test cases from evolving requirements and identify gaps, inconsistencies, and product risks.
• Collaborate with product managers, engineers, and AI specialists throughout the development lifecycle.
• Analyze defects, perform root cause analysis, and drive issue resolution.

Requirements

• Experience testing AI-powered applications, GenAI solutions, chatbots, copilots, or similar products.
• Understanding of AI/ML concepts and Large Language Models (LLMs).
• Strong analytical and problem-solving skills.
• Experience creating test plans, test cases, and quality metrics.
• Excellent spoken and written English.

Nice to have

• Experience with AI testing tools and frameworks.
• Experience evaluating LLM responses for accuracy, relevance, safety, and consistency.
• Familiarity with prompt engineering and AI quality assessment techniques.
• Knowledge of Python or another programming language used in automation.
• Experience working in Agile development environments.
• Exposure to cloud platforms such as Azure, AWS, or GCP.

Culture & Benefits

• Hybrid work arrangement with 3 office days per week in Sofia.
• Work within a diverse, equitable, and inclusive environment.
• Collaborate with a multinational team and specialists across engineering, product, and AI.

Hiring process

• Discover and apply.
• Interviews, assessments and tests, and background verification.
• Offer and onboarding.