Senior AI Engineer (LLM Evals)

Location: Must be based in San Francisco, CA (On-site, 5 days/week)

Compensation: $250,000 – $300,000 per year

Company

hirify.global is an AI startup building a highly personalized, persistent AI assistant that learns from user identity and behavior to perform real-world tasks across various tools.

What you will do

• Build a generalized evaluation system to measure assistant quality across multi-step agent trajectories.
• Develop golden datasets and a labeling loop to validate improvements and prevent regressions.
• Implement model routing and online evaluation tooling to optimize for cost, quality, and speed.
• Instrument quality metrics across the product to ensure every system change is measurable.
• Partner with engineering teams to close the loop from signal to fix.

Requirements

• Must be based in San Francisco and able to work 5 days a week in the Financial District office.
• Proven experience building or owning LLM evaluation systems or quality measurement at scale.
• Strong background in MLE or applied machine learning with a rigorous approach to measurement.
• Hands-on experience with the evaluation landscape, off-the-shelf tooling, and frameworks.
• Ability to reason about model routing and tradeoffs between different models.
• Senior or staff-level experience comfortable working in a greenfield environment.

Culture & Benefits

• Talent-dense team founded by former leaders from Plaid and Google.
• Well-funded startup backed by top-tier investors including Andreessen Horowitz and First Round Capital.
• Opportunity to own foundational 0-to-1 infrastructure for a high-impact AI product.