Senior Software Development Engineer In Test (ML/AI)

TL;DR

Senior Software Development Engineer In Test (ML/AI): Building and scaling automation frameworks for ML/AI quality, including model evaluation and LLM-assisted test generation with an accent on AI-powered workflows and distributed systems. Focus on designing scenario-based test suites for non-deterministic AI systems and integrating automated validation into CI/CD pipelines.

Location: United States, San Mateo, CA (Hybrid)

Salary: $183,000 - $274,600 USD

Company

Sony Interactive Entertainment (hirify.global) is a global leader in gaming and entertainment, delivering cutting-edge hardware and network services to millions of users.

What you will do

• Define quality strategies and test plans for ML-powered services and platform components.
• Use LLMs to generate and maintain high-value test cases for AI workflows.
• Design and develop scalable automation frameworks for backend services and ML inference systems using Python and Java.
• Build automated validation for LLM outputs, focusing on ranking, hallucinations, and probabilistic evaluation.
• Lead QE efforts for cross-functional projects, managing risks and release readiness.
• Integrate automated testing into CI/CD pipelines in collaboration with ML and engineering teams.

Requirements

• Bachelor’s degree in Computer Science or equivalent.
• 5+ years of experience as an SDET or QE engineer focused on backend and distributed systems.
• Experience using LLMs to generate test cases and utilizing AI evaluation tooling.
• Proficiency in Python, Java, or JS for automation development.
• Hands-on experience with frameworks like pytest, JUnit, Selenium, or Playwright.
• Experience with cloud and container technologies (AWS, GCP, Kubernetes, Docker).

Nice to have

• Experience validating ML outputs via statistical analysis or scenario-based testing.
• Familiarity with ML infrastructure platforms like Seldon, KServe, or Ray Serve.
• Background in content moderation ML, security, or fraud detection.
• Experience with Databricks or modern frontend technologies (React, Node.js).

Culture & Benefits

• Hybrid working policy.
• Top-tier benefits package including medical, dental, and vision insurance.
• Matching 401(k) and paid time off.
• Wellness program and employee discounts on Sony products.
• Collaborative environment focused on making play safer for everyone.