ML Evals Engineer (AI)

Location: In-person in San Francisco, California

Salary: $180,000–$350,000 plus equity

Company

hirify.global is an applied AI lab building large-scale web search infrastructure, embedding models, and vector databases.

What you will do

• Design and implement evaluation frameworks that probe the limits of search engines in an LLM-driven environment.
• Build scalable, reliable evaluation pipelines tracking regressions, drift, and quality signals across billions of documents.
• Create golden datasets, synthetic benchmarks, agentic tasks, and real-world test suites for developers, agents, and human users.
• Investigate how search quality should be evaluated and define measurement strategies for perfect search.
• Partner with ML researchers, data engineers, infrastructure engineers, and product teams to improve search-model feedback loops.

Requirements

• Hands-on ML experience in training, fine-tuning, or evaluating models.
• Strong engineering fundamentals and experience building reliable systems.
• Experience with Python, Rust, distributed pipelines, or GPU and cluster jobs.
• Ability to build evaluation sets, inspect edge cases, and develop creative measurement strategies.
• Willingness to work in person in San Francisco.

Nice to have

• Experience with embeddings or LLMs.

Culture & Benefits

• Premium medical, dental, and vision healthcare benefits.
• Fertility benefits.
• 16 weeks of fully paid parental leave for all new parents.
• Monthly wellness stipend.
• Visa sponsorship is available for international candidates, including STEM OPT, OPT, H1B, O1, and E3.