Senior AI QA Engineer

Location: Hybrid in Armenia

What you will do

• Develop and execute AI model evaluation strategies ensuring accuracy and fairness.
• Implement automated and manual testing for LLM-based applications.
• Collaborate with AI engineers to integrate testing and optimize model performance.
• Manage test datasets and develop synthetic data pipelines.
• Design frameworks for hallucination, bias, and robustness detection.
• Track AI performance metrics and maintain comprehensive test reports.

Requirements

• Experience with AI/ML testing frameworks and LLM evaluation methodologies.
• Strong understanding of LLM behaviors, biases, and failure modes.
• Proficiency in Python and ML testing tools.
• Strong English proficiency required.
• Experience with test dataset management and synthetic data generation.
• Strong problem-solving and debugging skills.

Culture & Benefits

• Equal opportunity employer with inclusive and respectful work environment.
• Collaborative team culture focused on AI-powered communication.