Staff Software Engineer - Agent Quality (AI)

TL;DR

Staff Software Engineer (AI Engineering): Designing and scaling the infrastructure, tooling, and developer workflows for evaluating AI agents with an accent on benchmarking, regression detection, and quality measurement. Focus on building a flywheel that connects evaluation results back into agent improvement across the full lifecycle.

Location: New York City, New York

Salary: $190,000 — $270,000 USD

Company

hirify.global is a data and AI company providing a Data Intelligence Platform to unify and democratize data, analytics, and AI for over 10,000 organizations.

What you will do

• Establish the foundational evaluation infrastructure for Genie Agents to enable rigorous benchmarking and quality measurement.
• Build a feedback flywheel that connects production signals and evaluation results directly back into agent improvement.
• Define the long-term technical direction for agent quality infrastructure as the platform expands.
• Collaborate with researchers and engineers to transform research ideas into robust internal platforms.
• Drive iterative development and training based on regression detection and quality metrics.

Requirements

• 6+ years of industry experience building software systems.
• Strong Python programming skills with experience in production or research infrastructure.
• Experience building and operating distributed systems, data pipelines, or large-scale infrastructure.
• Ability to design pragmatic and rigorous systems that produce trustworthy, reproducible signals.
• Experience working across ambiguous research and product boundaries.

Nice to have

• Experience with devtools, CI/CD platforms, testing frameworks, or observability tooling.
• Familiarity with LLM or agent quality measurement via evals or experimentation platforms.

Culture & Benefits

• Eligibility for annual performance bonuses and equity.
• Comprehensive benefits and perks tailored to the employee's region.
• Inclusive work environment committed to diversity and equal opportunity.