Software Development Engineer in Test, ML (AI)

TL;DR

Software Development Engineer in Test, ML (AI): Leading quality efforts for machine learning-powered products and services with an accent on validating probabilistic systems and scalable test automation. Focus on building confidence in model reliability, production readiness, and behavioral consistency through statistical signals and score distributions.

Location: Hybrid in San Mateo, CA, United States

Salary: $140,500 – $210,700 USD

Company

The company behind the hirify.global brand, delivering cutting-edge hardware and network services to millions of users globally.

What you will do

• Design and maintain automated test frameworks and scripts for ML inference services using Python or Java.
• Build representative test datasets to evaluate model behavior across various customer scenarios.
• Validate score distributions, ranking behavior, and output quality for incoming ML models.
• Perform functional, integration, regression, and API testing for microservices and databases.
• Collaborate with ML engineers and ops teams to ensure comprehensive CI/CD coverage for all features.
• Debug model anomalies and drive high quality standards for engineering decisions and customer features.

Requirements

• Bachelor's degree or equivalent.
• 3+ years of experience as an SDET focusing on backend systems.
• Proficiency in Python, Java, Scala, or Go for test automation.
• Hands-on experience with API testing (Postman), CI/CD pipelines (Jenkins, GitHub Actions, ArgoCD), and frameworks like pytest or JUnit.
• Experience with cloud and container technologies including AWS, GCP, Kubernetes, and Docker.
• Must be based in the United States to support the hybrid work model in San Mateo.

Nice to have

• Experience testing probabilistic ML systems such as recommendation or search ranking systems.
• Knowledge of data pipelines, feature stores, and model-serving infrastructure (Seldon, KServe, Ray Serve).
• Experience with performance and load testing tools like Locust, k6, Gatling, or JMeter.
• Experience with Databricks or similar ML platform tooling.

Culture & Benefits

• Comprehensive medical, dental, and vision insurance.
• Matching 401(k) plan and wellness program.
• Paid time off and exclusive employee discounts for Sony products.
• Inclusive environment that celebrates diversity and provides creative freedom.