Product Manager (AI)

TL;DR

Product Manager (AI): Lead the evaluations and leaderboard platform translating fast-moving AI research into trusted product infrastructure with an accent on multimodal evals, agentic workflows, reasoning traces, and benchmark categories. Focus on defining prototype-to-production pipelines, prioritizing evaluations based on frontier trends, and driving cross-functional execution for ecosystem adoption and credibility.

Bay Area

Company

The open platform for evaluating AI models in the real world, trusted by frontier labs, enterprises, and millions of users for leaderboards and human-centered model assessments.

What you will do

• Own the roadmap and product strategy for evaluations and leaderboard platform.
• Partner with ML researchers to productionize emerging evaluation methodologies like multimodal evals and agentic workflows.
• Drive prototype-to-launch workflows and cross-functional execution across research, engineering, design, and marketing.
• Prioritize evaluations based on model trends, developer demand, and ecosystem gaps.
• Build systems for evaluation quality, release cadence, and leaderboard credibility.
• Own metrics for adoption, engagement, and frontier-lab participation; engage users to shape future direction.

Requirements

• 5–8 years of product management in technical or ambiguous environments.
• Strong familiarity with AI systems including LLMs, multimodal models, agents, reasoning, and evaluation methodologies.
• Track record shipping complex products from concept to production.
• Experience translating research into product direction and execution.
• Strong systems thinking, cross-functional leadership, high agency, product judgment, and written communication.

Nice to have

• Technical background in CS, ML, or related.
• Experience in evaluations, benchmarking, AI infrastructure, or developer platforms.
• Prior products for researchers, ML engineers, or developers.
• Founder or early-stage startup experience.

Culture & Benefits

• Competitive compensation and equity based on location.
• Comprehensive health benefits including medical, dental, vision.
• Work on cutting-edge AI with small, mission-driven team valuing truth, speed, craftsmanship, curiosity, and impact.
• Culture of transparency, trust, excellence, energy, and focus; equal employment opportunities.