Member Of Technical Staff, LLM Evaluation (AI)

TL;DR

Member Of Technical Staff (AI): Developing and implementing methodologies to evaluate how well Copilot performs, focusing on task completion and affective aspects of the experience. Focus on building automated evaluation frameworks that help drive improvements in Copilot, track advances in research and adapt algorithms.

Location: New York, United States. Starting January 26, 2026, employees are expected to work from a designated hirify.global office at least four days a week if they live within 50 miles (U.S.) of that location.

Salary: USD $119,800 – $234,700 per year (Data Science IC4, US wide) or USD $158,400 – $258,000 per year (Data Science IC4, San Francisco Bay area and New York City metropolitan area), USD $139,900 – $274,800 per year (Data Science IC5, US wide) or USD $188,000 – $304,200 per year (Data Science IC5, San Francisco Bay area and New York City metropolitan area).

Company

hirify.global’s mission is to empower every person and every organization on the planet to achieve more.

What you will do

• Measure the performance of Copilot and identify failure modes and novel mitigation strategies.
• Create and implement comprehensive evaluation frameworks across diverse scenarios.
• Build automated testing systems and generalize solutions into repeatable frameworks.
• Maintain a user-oriented perspective by understanding needs from user perspectives and validating approaches through user research.
• Track advances in research and adapt algorithms to drive innovation in production systems.

Requirements

• Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 1+ year(s) data-science experience.
• OR Master’s Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 3+ years data-science experience.
• OR Bachelor’s Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5+ years data-science experience.
• Creative problem solving and independently shaping direction.
• Experience with prompt engineering and large language models.
• Experience writing production-quality Python code.

Nice to have

• Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5+ year(s) data-science experience.
• OR Master’s Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 7+ years data-science experience.
• OR Bachelor’s Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 10+ years data-science experience.
• Demonstrated interest in Responsible AI.

Culture & Benefits

• Employees come together with a growth mindset, innovate to empower others, and collaborate to realize shared goals.
• Values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.