Member of Technical Staff, Evaluations Engineering (AI)

TL;DR

Member of Technical Staff, Evaluations Engineering (AI): Building and optimizing evaluation infrastructure for generative AI on large-scale GPU clusters with an accent on developing sophisticated tools and techniques for reliability and performance. Focus on implementing state-of-the-art evaluation methods, inference strategies, and metrics algorithms for AI models.

Location: Redmond, United States. Employees are expected to work from a designated Microsoft office at least four days a week if they live within 50 miles of that location.

Salary: USD $139,900 – $331,200 per year.

Company

hirify.global is dedicated to advancing Copilot and other consumer AI products and research.

What you will do

• Develop and tune scalable software for Nvidia GB200 and AMD MIxxx architectures.
• Benchmark GPU clusters and gather data to develop the pretraining compute roadmap.
• Actively contribute to the development of AI models for innovative products.
• Design and build evaluation infrastructure for generative AI on large-scale GPU clusters.
• Collaborate with model scientists to implement state-of-the-art evaluation methods, inference strategies, and metrics algorithms.
• Drive architectural changes and influence the roadmap of relevant software and hardware components.

Requirements

• Bachelor’s Degree in Computer Science or a related technical field.
• 6+ years of technical engineering experience with coding in languages including C, C++, C#, Java, JavaScript, or Python.

Nice to have

• Master’s Degree in Computer Science or a related technical field AND 8+ years technical engineering experience OR Bachelor’s Degree AND 12+ years technical engineering experience.
• Experience with generative AI.
• Experience with distributed computing.
• Experience in leading technical projects and supporting architectural decisions with data.

Culture & Benefits

• Come together with a growth mindset, innovate to empower others, and collaborate to realize shared goals.
• Values of respect, integrity, and accountability to create a culture of inclusion.
• Opportunity to achieve more.