ML Engineer, Retrieval & Grounded Generation (AI)

Location: Remote, USA; occasional travel up to 25% to DEFCON AI HQ, customer sites, and vendor facilities.

Salary: $135,000–$170,000 per year.

Company

hirify.global is an incubation firm building technology-led companies across healthcare, cyber, and national security; DEFCON AI develops AI, mathematical optimization, data analytics, and software engineering solutions for resilient complex systems.

What you will do

• Own the system test plan across functional, integration, performance, and load testing, translating product acceptance criteria into executable tests.
• Build automated test suites, integrate them into CI/CD release gates, maintain regression coverage, and establish automated coverage standards.
• Create labeled evaluation sets, run sampled audits, measure incorrect decisions and user overrides, and report results independently from model builders.
• Maintain release inventories across builds, models, configurations, and thresholds, producing test reports, evaluation records, and rollback criteria.
• Coordinate customer user acceptance testing and verify that test environments match production closely enough for meaningful results.
• Hold the release gate and communicate when evidence is insufficient for shipment.

Requirements

• 5+ years in software quality engineering, test automation, or software engineering with substantial dedicated test ownership.
• Strong coding ability with a test-automation language and framework, such as Python with pytest or JavaScript/TypeScript with Playwright, Cypress, or Selenium; ability to read and reason about application code.
• Routine use of AI-assisted development, with judgment about where generated output requires verification.
• Experience building automated test suites, integrating them into CI/CD release gates, and owning a test plan through review and acceptance.
• Experience testing against requirements rather than implementation, including direct work with product owners or end users to define correctness.
• Experience with performance and load testing, working statistical literacy including sampling design, and experience measuring what a system misses.
• Ability to maintain independent measurement, gate work owned by non-reporting teams, hold releases when necessary, and produce clear audit-ready documentation.
• US citizenship required. Active US Secret clearance required from the start, along with a favorable investigation and CAC eligibility; elevated security requirements apply to some work.
• Willingness to travel up to 25% to customer sites, DEFCON AI HQ, and vendor facilities.

Nice to have

• Active Top Secret clearance.
• Government DevSecOps, RMF, or ATO experience and formal accreditation or audit evidence.
• Contract and API testing, test data management, and extensible test infrastructure.
• AI-assisted test generation and maintenance practices that keep generated tests meaningful.
• JMeter, k6, Locust, or comparable performance-testing experience.
• Machine-learning or model-output evaluation, including groundedness, citation checking, and analysis of user disagreement.
• Model cards, NIST AI RMF, responsible AI, Section 508, or WCAG verification experience.

Culture & Benefits

• Fully remote, results-based environment with flexible control over the workday.
• Competitive salary, bonus, and equity package.
• Employer-paid medical, dental, and vision insurance for employees and families.
• Unlimited PTO with manager approval and 14 weeks of fully paid parental leave.
• Optional 401(k), FSA, professional development opportunities, and mental health benefits.