Senior Software Development Engineer in Test (ML)

Location: Waterloo, Ontario, Canada, or Madison, Wisconsin, United States. The role may be subject to a hybrid working policy.

Base pay: CAD 114,000–143,000 annually.

Company

PlayStation builds technology products and tools used by millions of people worldwide, with engineering teams across four continents.

What you will do

• Lead architecture-level automation solutions and scalable quality engineering practices across multiple teams and product areas.
• Design, develop, and maintain automated test frameworks and scripts for ML inference services and platform components using Python, Java, Scala, or Go.
• Create realistic test accounts and datasets, and validate model score distributions, ranking behavior, output quality, and other signals before release.
• Perform functional, integration, regression, API, performance, and load testing for REST APIs, microservices, databases, and inference services.
• Partner with ML engineers and operations teams to ensure automated coverage for features and fixes and support continuous integration and deployment.
• Debug test failures and model anomalies, report defects, monitor test execution, and improve scalable testing practices.

Requirements

• Bachelor’s degree in Computer Science or a related field.
• At least 5 years of experience as an SDET, including strong experience testing backend systems.
• Proficiency in Python, Java, Scala, or Go for test automation.
• Hands-on experience with API testing tools such as Postman for distributed systems.
• Experience with CI/CD pipelines and Jenkins or similar tools, including GitHub Actions or ArgoCD.
• Experience building and maintaining automated test frameworks such as pytest, JUnit, or TestNG.
• Experience working with databases for backend validation.
• Experience with cloud and container technologies such as AWS, GCP, Kubernetes, or Docker.
• Familiarity with monitoring and observability tools such as Prometheus, Grafana, or Datadog, plus strong knowledge of SDLC, agile methodologies, problem-solving, and communication.

Nice to have

• Experience testing machine learning systems, including recommendation, search ranking, or other probabilistic applications.
• Experience creating synthetic or seeded test data and evaluating model outputs with statistical techniques, distribution analysis, or scenario-based validation.
• Knowledge of data pipelines, feature stores, inference systems, or model-serving infrastructure such as Seldon, KServe, or Ray Serve.
• Experience with performance and load testing tools, Databricks or similar ML platforms, and architecting CI/CD pipelines, quality gates, or release validation systems.

Culture & Benefits

• Collaborative engineering environment focused on partnership, technology, practical problem-solving, and customer-facing quality.
• Creative freedom, inclusive participation, and respect for diverse perspectives.
• Working arrangements and base pay may vary according to the hybrid working policy and job-related factors, including skills, experience, and location.