Senior Software Development Engineer in Test (ML/AI)

Location: Waterloo, Ontario, Canada; hybrid working policy applies.

Base pay: CAD 114,900–172,300 per year, plus potential bonus and benefits.

Company

Develop quality and security solutions for Sony Interactive Entertainment’s platforms, products, and player experiences.

What you will do

• Define quality strategies, test plans, and automation coverage for ML-powered services and platform components.
• Use LLMs and AI-assisted techniques to generate, expand, and prioritize test cases, including adversarial prompts, edge cases, ambiguous inputs, and underrepresented scenarios.
• Lead quality engineering for cross-functional ML projects, including risk assessment, dependency management, release readiness, and embedded QE delivery.
• Design and maintain scalable automation frameworks for backend services, REST APIs, microservices, and ML inference systems using Python and/or Java.
• Validate ML and LLM outputs through ranking behavior, score distributions, prompt and response quality, hallucination indicators, and probabilistic model evaluation.
• Perform functional, integration, regression, API, end-to-end, performance, and reliability testing; debug failures and integrate automation into CI/CD workflows.

Requirements

• Bachelor’s degree in Computer Science or equivalent.
• 5+ years of experience as an SDET or QE engineer focused on backend and distributed systems.
• Experience using LLMs to generate, transform, and prioritize test cases for AI-powered experiences.
• Experience with AI evaluation tooling, prompt evaluation frameworks, model monitoring, or human-in-the-loop review workflows.
• Strong experience testing RESTful APIs, microservices, and distributed architectures.
• Proficiency in Python, Java, JavaScript, or similar languages for automation development.
• Hands-on experience with pytest, JUnit, Selenium, Playwright, Cypress, or Appium.
• Experience with CI/CD systems and test pipelines such as Jenkins or GitHub Actions, plus cloud and container technologies such as AWS, GCP, Kubernetes, or Docker.
• Familiarity with databases, monitoring, observability tools, SDLC, Agile methodologies, and release processes.
• Excellent problem-solving, debugging, and communication skills.

Nice to have

• Experience validating ML outputs with statistical analysis or scenario-based testing.
• Familiarity with ML infrastructure, data pipelines, or model-serving platforms such as Seldon, KServe, or Ray Serve.
• Experience in content moderation ML, security, fraud detection, or adversarial ML.
• Experience testing high-scale, low-latency online services.
• Experience with Databricks or similar ML platform tooling.
• Familiarity with Node.js, React, or modern frontend technologies.
• Experience testing mobile, console, or other non-PC platforms.

Culture & Benefits

• Collaborative environment focused on curiosity, practical value, and cross-functional quality ownership.
• Medical, dental, and vision coverage.
• Matching 401(k), paid time off, and wellness program.
• Employee discounts on Sony products and potential bonus eligibility.