Product Data Scientist (AI)

TL;DR

Product Data Scientist (AI): Owning the evaluation loop for AI products including financial co-pilots and voice agents with an accent on offline eval suites and online quality monitoring. Focus on mining failure patterns from real traffic, hardening judge stability, and translating metrics into product decisions.

Location: Remote or hybrid within the EU

Company

A European fintech startup providing an all-in-one B2B financial solution for entrepreneurs, integrating banking, accounting, and invoicing.

What you will do

• Own and extend the offline evaluation suite, including datasets, judges, and metrics across products.
• Build and maintain online quality dashboards tracking resolution rates, CSAT, and LLM-as-judge signals.
• Analyze production failure patterns from real traffic to create regression cases and propose fixes.
• Improve evaluation methodology to handle judge stability and non-determinism.
• Translate data into actionable decisions through weekly syncs and clear trade-off analysis.

Requirements

• 3+ years of experience in analyst or data scientist roles, with at least one year in a product context.
• Proficiency in Python and SQL for end-to-end data analysis.
• Solid foundation in statistics, including sampling, hypothesis testing, and variance.
• Analytical mindset capable of starting from business questions rather than specific tools.
• Must be based in or able to work within the EU.

Nice to have

• Experience in quality analytics for ML systems such as ranking or recommendations.
• Hands-on experience evaluating LLM applications, including RAG, agents, and tool use.
• Personal projects or experience building LLM agents.

Culture & Benefits

• Stock Options Program available to all team members.
• AI-first engineering culture where AI-assisted coding (Claude Code) is the default.
• "Work & Swim" program offering one month in a corporate apartment in Cyprus.
• Flexible remote or hybrid work arrangements across Europe.
• Supportive, modern, and eco-conscious corporate environment.