AI Product Manager, AI Platform - Quality and Evaluation (AI)

Location: San Ramon, CA; Reno, NV; or New York, NY

Salary: $136,000–$170,000 typical starting range; $149,500–$187,000 in select locations including the San Francisco Bay Area and New York City Metro Area

Company

hirify.global develops enterprise software and AI capabilities for investment management workflows.

What you will do

• Define and execute the product strategy and roadmap for the AI Quality Platform.
• Build scalable LLM evaluation frameworks, automated evaluation harnesses, golden datasets, and quality benchmarks.
• Define AI observability capabilities for monitoring quality, latency, cost, reliability, and user outcomes in production.
• Partner with engineering and AI research teams on continuous regression testing and experimentation for generative AI systems.
• Use quality metrics, customer feedback, product analytics, and quantitative evaluation data to prioritize product investments.
• Communicate product vision, roadmap, and priorities across technical and executive stakeholders.

Requirements

• 5+ years of product management experience building highly technical platform, infrastructure, or developer-focused products.
• Experience shipping production AI products powered by large language models.
• Experience defining products involving LLM evaluation frameworks, AI quality measurement, and continuous model evaluation.
• Strong understanding of offline and online evaluation, model benchmarking, regression testing, and AI quality metrics.
• Experience with LLM observability platforms such as Datadog LLM Observability, Langfuse, LangSmith, Arize AI, Braintrust, or similar solutions.
• Must be authorized to work in the United States without employer sponsorship.

Nice to have

• Experience with RAG, AI agents, MCP, or model orchestration frameworks.
• Experience with synthetic data generation and automated evaluation pipelines.
• Experience building internal developer platforms or ML infrastructure products.
• Background in enterprise SaaS, developer platforms, or financial technology.

Culture & Benefits

• Ownership, transparency, positive intent, and a bias toward action.
• Collaborative, resilient, high-performing environment focused on continuous learning and experimentation.
• Unlimited vacation and educational and wellness reimbursements.
• $0 cost employee insurance plans.
• Company Stock Plan participation, subject to the applicable agreement.
• Opportunities for career advancement and meaningful product impact.