Senior/Expert Data Scientist (Conversational AI)

TL;DR

Senior/Expert Data Scientist (Conversational AI): Designing and implementing evaluation frameworks for a mission-critical emergency response platform with an accent on voice-based AI and LLM-based evaluation. Focus on building automated evaluation pipelines, ensuring AI safety, and connecting offline metrics with production business outcomes.

Location: Ukraine, Poland, Croatia, or elsewhere in Europe

Company

Custom software development company providing software engineering services and innovation partnership since 1991.

What you will do

• Design and implement evaluation frameworks for production conversational AI systems.
• Develop automated evaluation pipelines, benchmarks, and quality metrics for AI model assessment.
• Define methodologies for multi-turn conversations, dialogue flows, and voice interactions.
• Build representative datasets covering real-world scenarios, edge cases, and safety-critical conversations.
• Maintain regression testing frameworks for validating new model releases and platform updates.
• Analyze model failures and recommend improvements based on evaluation results.

Requirements

• 5+ years of experience in AI, Machine Learning, or Data Science.
• Proven experience designing evaluation frameworks for voice-based conversational AI or chatbots.
• Strong Python programming skills and experience building automated evaluation pipelines.
• Solid background in statistics, experiment design, and model benchmarking.
• Experience with both human and LLM-based evaluation methodologies.
• English: Upper-Intermediate (B2) or higher required.

Culture & Benefits

• Close cooperation with the customer.
• Opportunity to work on challenging tasks and influence project technologies.
• Continuous competence development.
• Professional team environment.
• Dynamic workplace with a low level of bureaucracy.