Senior Software Development Engineer in Test (AI/Agentic Quality Infrastructure)

Location: Austin, TX or Remote within the United States.

Company

hirify.global provides B2B real estate technology and a shared digital closing platform for homebuyers, sellers, lenders, title and escrow agents, and real estate agents.

What you will do

• Build and maintain production-grade test infrastructure, AI/ML evaluation systems, agentic validation harnesses, release orchestration tooling, observability platforms, and performance testing systems.
• Design agentic test harnesses that interact with applications, browsers, APIs, databases, logs, telemetry, and CI/CD systems to execute and validate complex workflows.
• Build evaluation frameworks for non-deterministic systems using deterministic assertions, semantic and model-based evaluation, regression datasets, human-in-the-loop review, and measurable quality signals.
• Develop structured logs, metrics, distributed tracing, execution timelines, and automated failure classification to accelerate root-cause analysis.
• Design test automation frameworks, migrate from Cypress to Playwright, and own test environment provisioning, parallelization, and CI/CD integration at scale.
• Partner with Software Engineering, Quality Engineering, SRE, and Platform teams, apply AI to test generation and release-risk analysis, and mentor engineers.

Requirements

• 5+ years of experience as an SDET or software engineer building test infrastructure, developer tooling, platform engineering, observability, or release engineering systems; experience should not be primarily focused on test cases or automation scripts.
• Strong programming ability in JavaScript and TypeScript, including backend services, libraries, frameworks, and production-quality infrastructure built from scratch.
• Strong software engineering fundamentals in system design, distributed systems, APIs, asynchronous systems, and production operations.
• Experience building or scaling CI/CD systems, release orchestration, test infrastructure, or developer platforms, including observability and telemetry with structured logging, metrics, and distributed tracing.
• Hands-on experience with LLMs, AI-assisted engineering, or agentic systems beyond using an AI coding assistant, including tool or function calling, agent workflows, and multi-step AI workflows.
• Experience or familiarity with LLM and agent evaluation, including evaluation datasets, deterministic and model-based evaluators, and testing probabilistic or non-deterministic systems.
• Strong understanding of the testing pyramid and experience selecting unit, integration, API, UI, performance, or agentic validation layers.
• Ability to mentor engineers and raise the technical quality bar across teams.

Nice to have

• No separate optional qualifications are specified.

Culture & Benefits

• Remote-first development organization with flexibility in how teams work.
• Comprehensive health plans, a 401(k) program, commuter benefits, parental leave, and flexible time off.
• Professional development support and a robust online onboarding program.
• Biweekly all-hands meetings and internal virtual events.
• Equal-opportunity workplace focused on diverse perspectives and respectful collaboration.