Senior Machine Learning Engineer, Agentic Evals and Self-Improvement (Nova)

Location: Remote within the United States; additional locations include Chicago, San Francisco, Atlanta, Austin, Boston, Denver, and New York.

Salary: $150K–$250K US base salary, determined by work location, skills, experience, education or training, and internal equity.

Company

hirify.global is an AI customer engagement platform for enterprise brands, with Nova as its intelligence layer for adapting customer experiences in real time.

What you will do

• Build evaluation systems for models and agents, including benchmarks, LLM judges, simulations, trajectory analysis, and regression tests.
• Create failure-diagnosis and improvement loops that turn evaluation findings into measurable fixes.
• Automatically optimize prompts, context, tools, rubrics, and agent harnesses.
• Build synthetic data pipelines for evaluation and post-training with attention to quality, privacy, and reproducibility.
• Apply current research such as LLM-as-judge, reward modeling, test-time search, and post-training to production systems.
• Mentor engineers and help set Nova’s technical direction.

Requirements

• 5+ years of hands-on machine learning engineering experience delivering complex ML systems into production at scale, including production ML pipelines or large-scale experimentation platforms.
• Master’s or PhD in Computer Science, Machine Learning, AI, or a related field.
• Strong Python skills and experience with modern ML frameworks; code must be readable, modular, and maintainable.
• Deep experience evaluating LLMs or agents, including multi-turn behavior, tool use, and reasoning.
• Experience building automated evaluation systems such as LLM judges, reward models, or benchmark infrastructure.
• Experience with at least one of prompt or context optimization, post-training, preference optimization, or reinforcement learning.
• Ability to apply current research by shipping reliable changes that measurably improve customer product experience.
• Strong communication and cross-team collaboration skills.

Nice to have

• Experience with LLM APIs from multiple providers, including Anthropic, OpenAI, and Google, as well as tool calling and prompt caching.
• Experience with automatic prompt optimization and error analysis.

Culture & Benefits

• Remote work is available within the United States.
• Work with autonomy across research, experimentation, and production systems.
• Growth mindset, ownership, collaboration, and transparency are core values.
• hirify.global supports a diverse and inclusive workplace and considers qualified applicants from all backgrounds.

Hiring process

• Apply through hirify.global’s official careers channels; legitimate recruiters will not request money, fees, or sensitive personal documents before an engagement letter or employment contract.