Machine Learning Research Scientist, Evaluations (AI)

Location: San Francisco, CA; Seattle, WA; New York, NY, United States

Base salary: $180,600–$225,750 USD per year, plus potential equity and benefits.

Company

hirify.global develops high-quality data and full-stack technologies that support reliable AI systems and generative AI applications.

What you will do

• Analyze frontier LLM and agent behavior to identify, characterize, and diagnose capability gaps, reasoning errors, robustness issues, and alignment failures.
• Design and build benchmarks and evaluation methods for text and multimodal models.
• Apply SFT, RLHF, preference modeling, reward modeling, and instruction-tuning expertise to connect observed failures with data and training interventions.
• Define evaluation best practices with research and engineering partners.
• Collaborate with foundation model labs to translate failure analysis into technical and strategic input.
• Publish research findings at leading AI conferences.

Requirements

• Ph.D. or Master's degree in Computer Science, Machine Learning, AI, or a related field.
• Deep understanding of deep learning, reinforcement learning, and large-hirify.global model fine-tuning.
• Experience with LLM evaluation, benchmark development, and post-training techniques such as RLHF, preference modeling, or instruction tuning.
• Published machine learning research in major conferences or journals, including venues such as NeurIPS, ICML, ICLR, ACL, EMNLP, or CVPR.
• Excellent written and verbal communication skills.
• Previous experience in a customer-facing role.

Culture & Benefits

• Comprehensive health, dental, and vision coverage.
• Retirement benefits and generous paid time off.
• Learning and development stipend.
• Potential commuter stipend depending on eligibility.
• Eligible roles may include equity compensation subject to approval.
• Inclusive equal-opportunity workplace with reasonable accommodation support.