Senior Machine Learning Engineer (LLM Evaluation & Agent Systems)

Location: San Jose, United States; fully in-person schedule up to 5 days a week

Base salary: $187,040–$438,000 annually, with potential additional bonuses, incentives, and restricted stock units.

Company

hirify.global protects the security, integrity, compliance, and privacy of TikTok data and the U.S. content ecosystem.

What you will do

• Design, build, and scale LLM evaluation architecture, including offline datasets, metric suites, and benchmarking pipelines.
• Develop synthetic data generation methods for data-scarce and long-tail risk scenarios.
• Build ML-driven risk decision models and reasoning abstractions from complex case data.
• Architect and optimize modular agent skills and tools for Data Science, Analytics, Data Engineering, and SRE workflows.
• Implement LLM-as-a-judge and multi-agent benchmarking methodologies.
• Provide technical leadership through ML system design patterns, benchmarking standards, and engineering best practices.

Requirements

• 5+ years of software or ML engineering experience designing, building, and deploying production ML/LLM systems.
• Hands-on expertise with LLM architectures, fine-tuning, RAG, prompt tuning, and agentic frameworks.
• Experience building LLM evaluation frameworks, automated benchmark pipelines, and solutions for data scarcity or imbalance.
• Strong coding skills and production experience with scalable data processing pipelines.
• Ability to provide technical leadership and translate operational requirements into scalable ML solutions.
• Ability to work fully in person in San Jose, with on-site presence up to 5 days per week.

Nice to have

• Experience in risk and compliance intelligence or automated decision systems.
• Experience with synthetic dataset generation or model bootstrapping for cold-start scenarios.
• Experience building AI tools or agents for data and engineering specialists.
• Experience serving as a founding technical lead for ambiguous, high-impact initiatives.

Culture & Benefits

• Medical, dental, and vision insurance from day one.
• 401(k) savings plan with company match.
• Paid parental leave, disability coverage, life insurance, and wellbeing benefits.
• 10 paid holidays, 10 paid sick days, and 17 days of Paid Personal Time.
• Inclusive workplace focused on creativity, curiosity, collaboration, and continuous iteration.