Software Development Test Engineer (SDET), AI & Automation

Location: 100% remote within the United States; hybrid work is available for candidates in the El Dorado Hills, California or Chicago, Illinois metro areas. Qualified candidates must be a U.S. Person: a U.S. citizen or lawful permanent resident.

Company

hirify.global develops cybersecurity software and the AI-enabled, cloud-native KeeperPAM platform for organizations and individual users worldwide.

What you will do

• Partner with AI engineers, software engineers and product managers to define test strategies, acceptance criteria and automation requirements.
• Design and maintain automated testing frameworks for AI applications, backend services, APIs and distributed workflows.
• Develop deterministic and probabilistic validation strategies, including evaluation criteria, tolerances and quality thresholds for AI and LLM functionality.
• Build regression, integration, API and end-to-end coverage, including test harnesses for prompts, models, agents and tool-calling workflows.
• Integrate automated validation and quality gates into CI/CD pipelines and maintain datasets, mocks, fixtures and synthetic data.
• Analyze failures, logs and telemetry; reduce flaky tests and improve testability, observability, reliability and maintainability.

Requirements

• 5+ years of experience in SDET, software development, test automation or code-intensive quality engineering.
• Hands-on experience testing AI/ML, LLM-powered or model-integrated systems and evaluating non-deterministic outputs.
• Strong programming skills in Python, Java, JavaScript/TypeScript, Rust or a similar general-purpose language.
• Experience building automated functional, integration, API and end-to-end test frameworks and integrating them with GitHub Actions, Jenkins, GitLab CI or similar tools.
• Experience testing REST APIs, distributed applications and Agile software teams; familiarity with TestRail, Git workflows, code review and debugging through logs and telemetry.
• Must be a U.S. Person, defined as a U.S. citizen or lawful permanent resident, due to FedRAMP compliance requirements.

Nice to have

• Experience with LLMs, agentic systems, retrieval-augmented generation, tool calling or prompt-driven workflows.
• Experience building model evaluation, benchmarking or regression frameworks and using AI evaluation metrics.
• Rust experience, performance or adversarial testing, and AI-assisted test or synthetic-data generation.
• Familiarity with Docker, Kubernetes, cloud-based testing environments or cybersecurity, identity and PAM software.

Culture & Benefits

• Embedded quality engineering collaboration across AI, engineering, product and QA.
• Medical, dental and vision insurance, including domestic partnerships.
• Employer-paid life insurance and supplemental life insurance options.
• Short- and long-term disability insurance options.
• 401(k) Roth and traditional plans and a generous PTO plan.