Sr. Software Dev Engineer, SageMaker AI

TL;DR

Sr. Software Dev Engineer, SageMaker AI (LLM/data preparation): Building scalable data quality and evaluation systems for machine learning workflows with an accent on auto-labeling, human-in-the-loop verification, and LLM-as-Judge techniques. Focus on architecting distributed ML infrastructure, establishing quality standards for AI agents and models, and continuously improving training data at human quality and machine scale.

Location: Onsite in Seattle, Washington, USA.

Salary: $168,100–$227,400 USD annually, plus sign-on payments, restricted stock units, and comprehensive benefits.

Company

Amazon Web Services develops and operates a broad cloud platform, including managed machine learning and data preparation services.

What you will do

• Design and deliver core data preparation components for customizing and fine-tuning large language models.
• Build scalable auto-labeling and LLM-as-Judge systems that detect, diagnose, and remediate data quality issues.
• Establish quality standards and evaluation frameworks for AI agents and machine learning models.
• Develop human-in-the-loop services for data labeling, annotation quality assurance, and ground-truth generation.
• Make architectural decisions across distributed systems, data processing, and ML infrastructure while owning the technical roadmap.
• Mentor engineers, lead design reviews, and influence technical direction across the team.

Requirements

• 5+ years of professional, non-internship software development experience.
• 5+ years of programming experience with at least one software programming language.
• 5+ years of leading the design or architecture of reliable and scalable systems.
• Experience mentoring engineers, serving as a tech lead, or leading an engineering team.
• Experience with the full software development life cycle, including coding standards, code reviews, source control, build processes, testing, and operations.
• Bachelor’s degree in computer science or equivalent experience.

Nice to have

• Experience building ML pipelines, data processing systems, or evaluation infrastructure at scale.
• Hands-on experience with LLM prompting, fine-tuning, structured output, and confidence calibration.

Culture & Benefits

• Knowledge-sharing, mentorship, and thorough code reviews support engineering growth.
• Flexible working practices support work-life harmony.
• Benefits include medical, dental, vision, prescription, life and AD&D insurance, mental health support, flexible spending accounts, and 401(k) matching.
• Paid time off, parental leave, and adoption and surrogacy reimbursement are available.
• Employee-led affinity groups and inclusion-focused learning events support a diverse workplace.