Member Of Technical Staff - Data Quality Engineer (Pre-Training)

TL;DR

Member of Technical Staff - Data Quality Engineer (Pre-training): Ensuring the quality, reliability, and downstream impact of data used to train AI models with an accent on translating requirements into measurable quality signals and providing actionable feedback to external data vendors. Focus on designing, validating, and scaling automated QA methods to reliably measure data quality across large campaigns.

Location: Onsite in San Francisco, London, or New York

Company

Reflection’s mission is to build open superintelligence and make it accessible to all.

What you will do

• Own upstream data quality for LLM pre-training across languages and modalities.
• Partner with research and pre-training teams to translate requirements into measurable quality signals and provide actionable feedback to external data vendors.
• Design, validate, and scale automated QA methods to reliably measure data quality across large campaigns.
• Build reusable QA pipelines that reliably deliver high-quality data to pre-training teams for model training.
• Monitor and report on data quality over time, driving continuous iteration on quality standards, processes, and acceptance criteria.

Requirements

• Strong engineering fundamentals with experience building data pipelines, QA systems, or evaluation workflows for pre-training data.
• Detail-oriented with an analytical mindset, able to identify failure modes, inconsistencies, and subtle issues that affect data quality.
• Solid understanding of how data quality impacts pre-training, with the ability to translate quality concerns into concrete signals, decisions, and feedback.
• Experience designing and validating automated quality checks, including rule-based systems, statistical methods, or model-assisted approaches such as LLM-as-a-Judge.
• Proficiency in Python and building ML / LLM workflows. Must be comfortable debugging and writing scalable code.
• Excellent communication skills with the ability to clearly articulate complex technical concepts across teams.

Culture & Benefits

• Top-tier compensation with salary and equity.
• Comprehensive medical, dental, vision, life, and disability insurance.
• Fully paid parental leave for all new parents, including adoptive and surrogate journeys, and financial support for family planning.
• Paid time off, relocation support, and other perks.
• Opportunities to connect with teammates with daily lunch and dinner provided, regular off-sites, and team celebrations.