Senior Test Engineer - Metrics Quality

Location: Austin, Texas, United States; onsite. Candidates must be authorized to work in the U.S.; remote work is not available.

Company

hirify.global develops autonomous driving software and uses recorded-scene metrics to support release decisions.

What you will do

• Own acceptance of new and changed metrics before they are used for release decisions.
• Define and maintain comprehensive case spaces, including scenarios where metrics must remain silent.
• Investigate silent failures and identify situations that metrics fail to report.
• Build targeted test data, analysis scripts, and independent reference computations.
• Present concrete findings to metric owners, including defect frequency and decision impact.
• Qualify metric releases and maintain regression coverage for semantic changes as the metric portfolio grows.

Requirements

• 6+ years of experience in software testing or test engineering, with depth in data-heavy or analytical systems.
• Strong Python and SQL skills for analysis scripts and independent reference computations.
• Statistical literacy covering distributions, variance, sample size, aggregation traps, and distinguishing noise from meaningful differences.
• Product-oriented testing approach and the ability to challenge expected metric behavior with evidence.
• Fluent use of LLMs for analysis, test-data generation, unfamiliar code review, and reasoning cross-checks.
• Clear written English and authorization to work in the United States are required.

Nice to have

• Experience with autonomous vehicles, robotics, or machine-learning evaluation.
• ClickHouse or a similar analytical database.
• Experience testing evaluation pipelines or A/B testing infrastructure.
• Experience designing annotation tasks for human reviewers and checking answer agreement.
• Knowledge of U.S. road rules and driver behavior.

Culture & Benefits

• Metrics are released on their own cadence and qualified through a dedicated release cycle.
• Work includes close collaboration with engineers who own the metrics.
• hirify.global provides reasonable accommodations for qualified applicants and employees with disabilities.
• The employer does not offer relocation or visa sponsorship.