AI Engineer

Location: San Francisco, United States; on-site five days per week

Salary: $300K–$350K annually, plus equity

Company

hirify.global builds agentic AI products that automate critical work across the insurance industry.

What you will do

• Build evaluation frameworks and infrastructure for measuring the accuracy, quality, and reliability of AI agents.
• Develop golden datasets, benchmarks, and regression suites for real-world insurance workflows and edge cases.
• Design evaluations for multi-step agent trajectories, tool use, reasoning, and end-to-end task completion.
• Create automated grading systems using LLM-as-judge methods, deterministic graders, human evaluation workflows, and quality rubrics.
• Integrate offline and online evaluations into development and release processes to detect regressions and measure improvements.
• Benchmark frontier and open-weight models across accuracy, latency, cost, and reliability, using results to guide production model selection.

Requirements

• 5+ years of software engineering experience with hands-on experience building production AI/ML systems.
• Experience with evaluation frameworks, LLM applications, model benchmarking systems, or applied AI infrastructure.
• Strong understanding of modern LLMs, agent architectures, evaluation methodologies, and model performance tradeoffs.
• Strong Python engineering skills and the ability to build reliable production-grade systems.
• Ability to independently own ambiguous technical problems from research and experimentation through production.
• Clear written and verbal communication skills.

Nice to have

• Experience with LLM-as-judge systems, human evaluation, agent trajectory evaluations, or large-scale benchmarking.
• Experience with open-weight models, fine-tuning, post-training, or model routing.
• Background in B2B SaaS or 0-to-1 product development.

Culture & Benefits

• Highly technical environment with significant engineering influence on product direction.
• Fully covered health, dental, and vision benefits.
• Meaningful stock options and a company 401(k).
• Unlimited paid time off and pre-tax commuter benefits.
• On-site lunch and dinner, team events, and off-sites.