SDET Contractor (Agentic AI)

Location: Aliso Viejo, California, United States; onsite contractor role

Salary: $58–$72 USD per hour

Company

hirify.global creates and supports PlayStation hardware, network services, and entertainment experiences for more than 100 million people as part of Sony Group Corporation.

What you will do

• Develop QE strategies, test plans, automated frameworks, quality metrics, and release-readiness criteria for Agentic AI systems, backend services, APIs, and integrations.
• Test Agentic AI workflows including multi-step execution, tool selection and invocation, task completion, memory or state interactions, and failure handling.
• Evaluate AI-generated responses for accuracy, relevance, groundedness, consistency, and expected behavior beyond exact-match assertions.
• Validate AI guardrails, unexpected inputs, incorrect tool usage, prompt-based attacks, API contracts, schemas, authentication, business rules, and error handling.
• Integrate automation into CI/CD pipelines, establish quality gates, and apply shift-left and risk-based testing throughout development.
• Investigate distributed-system failures, analyze defects and production issues, perform root-cause analysis, and support triage, regression testing, and release readiness.

Requirements

• Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field, or equivalent practical experience.
• 4+ years of experience in SDET, Quality Engineering, Software Engineering, or a related discipline.
• Strong experience testing REST APIs, backend services, microservices, distributed systems, and service-to-service integrations.
• Hands-on experience developing automated tests with Python, Java, JavaScript, TypeScript, or a similar programming language.
• Experience with functional, integration, regression, contract, and end-to-end testing, including designing and maintaining automated test frameworks.
• Experience integrating automated tests into CI/CD pipelines and working with Git and CI/CD development workflows.
• Hands-on experience testing Generative AI, LLM-powered applications, or Agentic AI systems, including non-deterministic behavior and output evaluation beyond exact-match assertions.
• Strong understanding of software testing methodologies, Quality Engineering practices, risk-based test strategies, testing layers, analytical debugging, and root-cause analysis.
• Experience working in an Agile development environment and collaborating across engineering teams.

Nice to have

• Experience testing Agentic AI planning, multi-step execution, tool calling, memory, and state management.
• Experience with evaluation datasets, semantic evaluation, LLM-as-a-Judge, groundedness, relevance, and hallucination detection.
• Experience with LangSmith, DeepEval, Ragas, Arize Phoenix, Promptfoo, or similar AI evaluation and observability tools.
• Experience testing RAG systems and AI agents interacting with APIs, databases, or external tools.
• Experience with Pact, OpenAPI/Swagger, event-driven architectures, Kafka, Docker, Kubernetes, AWS, or other cloud platforms.
• Experience with performance, scalability, reliability, resiliency testing, QE metrics, quality gates, or cross-team testing standards.

Culture & Benefits

• Onsite work in Aliso Viejo, California, United States.
• Benefits include medical, dental, and vision offerings.
• Inclusive, equal-opportunity, and Fair Chance workplace.
• Background checks are conducted at the offer stage for new employees.