Software Development Engineer In Test (ML/AI)

TL;DR

Software Development Engineer In Test (ML/AI): Building and optimizing automation frameworks for ML-powered services and LLM evaluation with an accent on model validation, AI-assisted test generation, and distributed systems. Focus on designing scenario-based test suites for non-deterministic AI systems and integrating automated testing into CI/CD pipelines.

Location: Hybrid in San Mateo, CA, United States

Salary: $140,500 - $210,700 USD

Company

hirify.global (Sony Interactive Entertainment) is a global entertainment leader delivering cutting-edge hardware and network services to over 100 million people.

What you will do

• Define quality strategies, test plans, and automation coverage for ML-powered services and platform components.
• Utilize LLMs and AI-assisted techniques to generate and maintain high-value test cases for AI workflows.
• Design and maintain scalable automation frameworks for backend services and ML inference systems using Python and Java.
• Develop automated validation for LLM outputs, including ranking, prompt quality, and hallucination indicators.
• Lead QE efforts for cross-functional projects, driving risk assessment and release readiness.
• Partner with engineering and ML teams to integrate automated testing into CI/CD pipelines.

Requirements

• Bachelor’s degree in Computer Science or equivalent practical experience.
• 3+ years of experience as an SDET or QE engineer focused on backend and distributed systems.
• Experience using LLMs to generate and prioritize test cases for AI-powered experiences.
• Proficiency in Python, Java, or JS for automation development.
• Hands-on experience with automation frameworks (pytest, JUnit, Selenium, Playwright, or Cypress).
• Experience with cloud and container technologies including AWS, GCP, Kubernetes, and Docker.

Nice to have

• Experience validating ML outputs using statistical analysis or scenario-based testing.
• Familiarity with ML infrastructure (Seldon, KServe, Ray Serve).
• Prior work in content moderation ML, security, fraud detection, or adversarial ML.
• Experience with Databricks or testing high-scale, low-latency online services.
• Familiarity with Node.js, React, or modern frontend technologies.

Culture & Benefits

• Top-tier benefits package including medical, dental, and vision insurance.
• Matching 401(k) plan.
• Paid time off and wellness programs.
• Employee discounts for Sony products.
• Inclusive environment that values diversity and fosters curiosity for innovation.