Product Manager - AI Evaluations Tooling

Location: Illinois, United States; remote/hybrid

Salary: $115,000–$173,000 annually, plus an annual performance bonus and long-term incentives.

Company

hirify.global develops a portfolio of AI products and legal technology solutions.

What you will do

• Own the product vision, strategy, and roadmap for the AI Evaluations Platform.
• Develop offline and online evaluation pipelines, production monitoring, eval discovery and reuse, SME authoring and approval, and agentic evaluation capabilities.
• Conduct continuous discovery with applied scientists, application engineering teams, product managers, and other technical customers.
• Lead the development of agentic testing for trajectory evaluation, tool-call correctness, intermediate-state rubrics, long-running judge orchestration, and multi-step flow simulation.
• Build team-facing quality dashboards and workflows that help AI product teams measure performance and understand release changes independently.
• Lead build-vs-buy evaluations, vendor reviews, judge calibration, and communication of technical tradeoffs to technical and non-technical stakeholders.

Requirements

• 4+ years of product management experience in a technology company, including at least 2 years with technical, platform, infrastructure, or data/ML products.
• Experience with LLM-based products and AI evaluation concepts such as rubrics, LLM-as-judge, and offline versus online evaluation.
• Experience with platform, developer-tools, or data/ML products serving technical customers, or the ability to quickly build credibility in a technical domain.
• Experience conducting user discovery with technical audiences and translating findings into roadmap decisions.
• Solid understanding of the software development lifecycle and agile practices, with strong analytical, written, and verbal communication skills.
• Role location: Illinois, with a remote/hybrid work arrangement.

Nice to have

• Direct experience with agentic systems.
• Experience translating subject-matter-expert judgment into workflows that can be encoded, scored, and reused.

Culture & Benefits

• Work on a shared AI platform used by teams across a portfolio of 20+ AI products.
• Competitive base salary with an annual performance bonus and long-term incentives.
• Opportunity to work directly with applied scientists, engineers, product managers, legal experts, and executive stakeholders.