Tech Lead Machine Learning Engineer (LLM Evaluation & Agent Systems)

Location: Seattle, United States; fully in-person schedule up to 5 days a week

Base salary: $198,360–$416,100 annually

Company

hirify.global operates data privacy, cybersecurity, trust and safety, and compliance programs for U.S. user data and apps.

What you will do

• Design, build, and scale LLM evaluation architecture, including offline datasets, metric suites, and benchmarking pipelines.
• Develop synthetic data generation methods for data-scarce and long-tail risk scenarios.
• Build ML-driven decision models and reasoning abstractions from complex risk and compliance case data.
• Architect and optimize modular agent skills and tools for Data Science, Analytics, Data Engineering, and SRE workflows.
• Implement LLM-as-a-judge and multi-agent benchmarking methodologies.
• Set ML system design patterns, benchmarking standards, and engineering best practices while leading cross-functional delivery.

Requirements

• 5+ years of software or ML engineering experience designing, building, and deploying production ML/LLM systems.
• Hands-on expertise with LLM architectures, fine-tuning, RAG, prompt tuning, and agentic frameworks.
• Experience building LLM evaluation frameworks, automated benchmark pipelines, and solutions for data scarcity or imbalance.
• Strong coding skills and production experience with scalable data processing pipelines.
• Ability to work on-site in Seattle, with an in-person schedule of up to 5 days per week.

Nice to have

• Experience in risk and compliance intelligence or automated decision systems.
• Experience generating synthetic datasets or bootstrapping models for cold-start scenarios.
• Experience building AI tools or agents for Data Analysts, Data Scientists, Data Engineers, or Data SREs.
• Experience serving as a founding technical lead for ambiguous, high-impact initiatives.

Culture & Benefits

• Medical, dental, and vision insurance from the first day.
• 401(k) savings plan with company match.
• Paid parental leave, disability coverage, life insurance, and wellbeing benefits.
• 10 paid holidays, 10 paid sick days, and 17 days of paid personal time.
• Inclusive, collaborative environment focused on curiosity, learning, resilience, and continuous iteration.