Senior Software Development Engineer In Test (ML/AI)

TL;DR

Senior Software Development Engineer In Test (ML/AI): Developing and implementing quality strategies and automation frameworks for ML-powered services and AI-powered workflows with an accent on model evaluation, LLM-assisted test generation, and validation of non-deterministic AI outputs. Focus on designing scenario-based test suites, building automated validation for LLM behaviors, and coordinating quality initiatives across cross-functional teams.

Location: Hybrid (United States)

Salary: $183,000 — $274,600 USD

Company

hirify.global (SIE) is a global leader in gaming, focusing on creating safe and amazing player experiences through its platforms and products.

What you will do

• Define quality strategies, test plans, and automation coverage for ML-powered services and platform components.
• Utilize LLMs and AI-assisted techniques to generate and maintain high-value test cases for ML workflows.
• Design and develop scalable automation frameworks for backend services and ML inference systems using Python and/or Java.
• Build automated validation for LLM outputs, focusing on hallucination indicators, ranking behavior, and probabilistic evaluation.
• Lead QE efforts for multi-functional projects, driving risk assessment and release readiness.
• Integrate automated testing into CI/CD pipelines and collaborate on scalable quality standards and tooling.

Requirements

• Bachelor’s degree in Computer Science or equivalent.
• 5+ years of experience as an SDET or QE engineer focusing on backend and distributed systems.
• Experience using LLMs for test case generation and utilizing AI evaluation tooling or model monitoring.
• Proficiency in Python, Java, or JS for automation development.
• Hands-on experience with frameworks such as pytest, JUnit, Selenium, Playwright, or Cypress.
• Experience with AWS, GCP, Kubernetes, Docker, and CI/CD systems (Jenkins, GitHub Actions).

Nice to have

• Experience validating ML outputs using statistical analysis or scenario-based testing.
• Familiarity with ML infrastructure (Seldon, KServe, Ray Serve) or Databricks.
• Prior experience in content moderation ML, security, or fraud detection.
• Experience testing high-scale, low-latency online services or non-PC platforms (mobile, console).

Culture & Benefits

• Hybrid working policy.
• Top-tier benefits package including medical, dental, and vision insurance.
• Matching 401(k) retirement plan.
• Paid time off and wellness programs.
• Employee discounts for Sony products.