Sr Software Development Engineer In Test (ML)

TL;DR

Sr Software Development Engineer In Test (ML): Designing and implementing automated test frameworks for ML inference services and platform components with an accent on validating probabilistic systems and model reliability. Focus on building scalable validation approaches using statistical signals and score distributions to ensure production readiness at global scale.

Location: United States, San Mateo, CA (Hybrid)

Salary: $183,000 - $274,600 USD

Company

Sony Interactive Entertainment is the company behind the hirify.global brand, delivering cutting-edge hardware and network services to millions of users globally.

What you will do

• Design, develop, and maintain automated test frameworks and scripts for ML inference services using Python and Java.
• Build and automate representative test datasets and account creations to evaluate model behavior across various scenarios.
• Validate score distributions, ranking behavior, and output quality for incoming ML models prior to release.
• Perform functional, integration, regression, and API testing for microservices and RESTful APIs.
• Conduct non-functional performance and load testing to verify inference service scalability and latency requirements.
• Collaborate with ML engineers and ops teams to ensure automated test coverage is integrated into CI/CD pipelines.

Requirements

• Bachelor's degree or equivalent.
• 5+ years of experience as an SDET with a strong focus on testing backend systems.
• Proficiency in Python, Java, Scala, or Go for test automation.
• Hands-on experience with API testing tools (e.g., Postman) and CI/CD pipelines (Jenkins, GitHub Actions, ArgoCD).
• Experience building automated test frameworks (pytest, JUnit, TestNG) and working with databases for backend validation.
• Experience with cloud and container technologies (AWS, GCP, Kubernetes, Docker) and monitoring tools (Prometheus, Grafana, Datadog).

Nice to have

• Experience testing probabilistic ML systems, such as recommendation or search ranking systems.
• Experience evaluating model outputs using statistical techniques, distribution analysis, or scenario-based validation.
• Knowledge of data pipelines, feature stores, or model-serving infrastructure (Seldon, KServe, Ray Serve).
• Experience with performance and load testing tools (Locust, k6, Gatling, JMeter).
• Experience with Databricks or similar ML platform tooling.

Culture & Benefits

• Top-tier benefits package including medical, dental, and vision insurance.
• Matching 401(k) retirement plan.
• Paid time off and wellness programs.
• Coveted employee discounts for Sony products.
• Inclusive and diverse engineering environment with teams across four continents.