Senior Product Manager (AI)

TL;DR

Senior Product Manager (AI): Defining and maintaining an open, community-driven benchmark for evaluating AI agents and IDE-embedded AI features at scale with an accent on evaluation methodology, task design, and metrics. Focus on translating real-world coding tasks into representative evaluation workloads and driving business and ecosystem development through external partnerships.

Location: Hybrid role, based in Amsterdam, Belgrade, Berlin, Limassol, Madrid, Munich, Paphos, Prague, Tel Aviv, Warsaw, or Yerevan.

Company

hirify.global is a product company passionate about code, building strong developer tools, and seeing the next frontier in AI-powered developer tools.

What you will do

• Refine the criteria for what makes an evaluation task representative for software engineers and IDE AI features.
• Define and iterate the evaluation methodology, designing tasks, specifying metrics, and success criteria.
• Build and maintain relationships with external vendors, platform partners, and internal stakeholders.
• Curate and manage datasets, coordinating contributions from various sources to maintain quality and diversity.
• Own the DPAI Arena evaluation standard and roadmap, aligning with the broader hirify.global AI and IDE strategy.

Requirements

• Proven experience in product management (developer tools or data-driven products is a plus).
• Experience working on GenAI features for developer tooling.
• Strong analytical skills to define meaningful metrics and translate results into product design.
• Experience working with multiple internal and external stakeholders.
• Understanding of developer workflows, IDEs, and AI-assisted development.
• Strong communication skills, both written and spoken.
• Ownership mindset and proactive approach.
• Engineering background (particularly SWE or QA for tech products).

Nice to have

• Experience maintaining or curating open-source products.
• ML experience.
• Familiarity with outsourcing or crowdsourcing platforms, data collection pipelines, or community-driven ecosystems.

Culture & Benefits

• Work on defining the next frontier in AI-powered developer tools.
• Contribute to an open, community-driven benchmark for AI evaluation.
• Work in a company passionate about code and effective developer tools.
• Opportunity to grow, discover, and create by automating routine checks.