SDET Contractor (Agentic AI)

Location: United States, Aliso Viejo, CA; other location: Ontario, Canada

Salary: USD 58–72 per hour

Company

hirify.global is the entertainment brand of Sony Interactive Entertainment, delivering gaming hardware, network services, and interactive entertainment experiences to more than 100 million people.

What you will do

• Develop QE strategies and test plans for Agentic AI systems, backend services, APIs, and service integrations.
• Design and maintain scalable automation for API, functional, integration, contract, regression, and end-to-end testing.
• Test Agentic AI workflows, including multi-step execution, tool selection, task completion, failure handling, guardrails, and prompt-based attacks.
• Evaluate AI-generated responses for accuracy, relevance, groundedness, consistency, and expected behavior beyond exact-match assertions.
• Validate API contracts, schemas, authentication, business rules, error handling, and interactions among agents, tools, data sources, and downstream services.
• Integrate testing into CI/CD, investigate distributed-system failures, analyze root causes, and report quality metrics and release readiness.

Requirements

• Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field, or equivalent practical experience.
• 4+ years of experience in SDET, Quality Engineering, Software Engineering, or a related discipline.
• Strong experience testing REST APIs, backend services, microservices, distributed systems, and service-to-service integrations.
• Hands-on experience developing automated tests with Python, Java, JavaScript, TypeScript, or a similar programming language.
• Experience with functional, integration, regression, contract, and end-to-end testing.
• Experience integrating automated tests into CI/CD pipelines and using Git development workflows.
• Strong understanding of software testing methodologies, Quality Engineering, risk-based testing, and appropriate testing layers.
• Hands-on experience testing Generative AI, LLM-powered applications, or Agentic AI systems, including non-deterministic behavior and output evaluation beyond exact matches.
• Strong analytical, debugging, and root-cause-analysis skills; experience working in an Agile environment and collaborating across engineering teams.

Nice to have

• Experience testing Agentic AI planning, multi-step execution, tool calling, memory, or state management.
• Experience with evaluation datasets, semantic evaluation, LLM-as-a-Judge, groundedness, relevance, or hallucination detection.
• Experience with LangSmith, DeepEval, Ragas, Arize Phoenix, Promptfoo, or similar AI evaluation and observability tools.
• Experience testing RAG systems or AI agents interacting with APIs, databases, or external tools.
• Experience with Pact, OpenAPI/Swagger, event-driven architectures, Kafka, Docker, Kubernetes, AWS, or similar platforms and tools.
• Experience with performance, scalability, reliability, resiliency testing, QE metrics, quality gates, or cross-team testing standards.

Culture & Benefits

• Inclusive environment focused on diversity, employee empowerment, and equal opportunity.
• Medical, dental, and vision benefit offerings.
• Background checks are conducted at the offer stage for new employees.