Senior Quality Automation Engineer (AI)

Location: Napier, New Zealand, or remote from another city in New Zealand. Hybrid and remote-friendly work options are available.

Company

Re-Leased is a New Zealand SaaS company building am:pm, an agentic AI product for commercial real estate operators.

What you will do

• Own quality automation across the test pyramid and steward the end-to-end test suite.
• Design pipeline gates, smoke checks, release signals, and other automated controls supporting continuous delivery.
• Use AI tools for test generation, failure triage, and test data creation while reviewing and refining generated results.
• Shape testing practices for AI-powered features, including evaluations of non-deterministic outputs, tool use, and statistical correctness.
• Partner with engineers on acceptance criteria and user-focused positive, negative, and edge-case scenarios.
• Run exploratory and risk-based testing, analyse quality signals, and feed production defects back into automation.

Requirements

• 5+ years of QA experience in a SaaS product environment with automation central to the work.
• Experience designing automation frameworks from the ground up and building JavaScript/TypeScript end-to-end suites with tools such as Playwright or Cypress.
• Strong experience with Git, branches, pull requests, code review, and reading production code.
• Fluency in CI/CD and experience defining pipeline gates, smoke checks, and rollback signals for safe continuous delivery.
• Current practical use of AI tools and a strong interest in testing non-deterministic AI systems and evaluations.
• Strong exploratory testing skills, test-pyramid judgment, customer focus, and comfort working in ambiguous early-stage product environments.

Nice to have

• Experience with LLM evaluation frameworks, golden datasets, judge models, and statistical thresholds.
• Experience with adversarial AI testing, including prompt injection, jailbreaks, and hostile inputs.
• REST API testing, performance testing, or accessibility testing experience.
• Start-up or zero-to-one product experience.

Culture & Benefits

• Small-team start-up environment within an established SaaS business, with high autonomy and close customer feedback.
• Flexible hybrid and remote-friendly work supporting teams across New Zealand, Australia, the United Kingdom, and North America.
• Health insurance, premium employee assistance services, and an annual wellness allowance.
• Generous paid parental leave and family-friendly support.
• Milestone rewards and additional leave linked to tenure and contribution.