SDET Contractor (Agentic AI)

Location: Ontario, Canada; the role follows a hybrid working policy.

Salary: CAD 51.61–77.42 per hour.

Company

hirify.global develops PlayStation hardware, network services, and interactive entertainment experiences for a global audience.

What you will do

• Develop QE strategies and test plans for Agentic AI systems, backend services, APIs, and integrations.
• Design and maintain automated frameworks for functional, integration, contract, regression, and end-to-end testing.
• Test multi-step AI workflows, tool selection and invocation, task completion, memory, state management, and failure handling.
• Evaluate AI-generated responses for accuracy, relevance, groundedness, consistency, and expected behavior.
• Test guardrails, unexpected inputs, prompt-based attacks, API contracts, authentication, schemas, and service integrations.
• Investigate distributed-system failures, analyze root causes, establish quality metrics, and support defect triage and release readiness.

Requirements

• Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field, or equivalent practical experience.
• 4+ years of experience in SDET, Quality Engineering, Software Engineering, or a related discipline.
• Strong experience testing REST APIs, backend services, microservices, distributed systems, JSON, schemas, authentication, and error handling.
• Hands-on experience developing automated tests with Python, Java, JavaScript, TypeScript, or a similar language.
• Experience with functional, integration, regression, contract, and end-to-end testing, including CI/CD integration, Git, and Agile development.
• Hands-on experience testing Generative AI, LLM-powered, or Agentic AI systems and evaluating non-deterministic outputs beyond exact-match assertions.

Nice to have

• Experience with AI evaluation techniques, evaluation datasets, semantic evaluation, LLM-as-a-Judge, groundedness, relevance, and hallucination detection.
• Experience with LangSmith, DeepEval, Ragas, Arize Phoenix, Promptfoo, Pact, or OpenAPI/Swagger.
• Experience testing RAG systems, event-driven architectures, Kafka, Docker, Kubernetes, AWS, or other cloud platforms.
• Experience with performance, scalability, reliability, resiliency testing, QE metrics, quality gates, or cross-team testing standards.

Culture & Benefits

• Hybrid working policy in Ontario.
• Inclusive and diverse work environment with equal-opportunity employment practices.
• Background checks are conducted at the offer stage.

Hiring process

• Background checks may include criminal record checks for some roles and are completed at the offer stage.