MTS Data Engineer (AI)

TL;DR

MTS Data Engineer (AI): Building and optimizing high-scale data ETL pipelines and experimentation frameworks for Copilot with an accent on data quality, schema management, and real-time processing. Focus on architecting scalable data infrastructure for machine learning model training and inference and collaborating with ML engineers.

Location: Required to be local to the San Francisco area or Redmond area and in office 3 days a week.

Salary: USD $139,900 – $331,200 per year

Company

hirify.global is empowering every person and every organization on the planet to achieve more.

What you will do

• Build, maintain, and enhance data ETL pipelines for large-scale data with low latency and high throughput to support Copilot.
• Design and maintain experimentation reporting pipelines for measuring model performance and user engagement.
• Own data quality initiatives including monitoring, alerting, validation, and remediation processes.
• Implement robust schema management solutions for quick and seamless schema evolution.
• Develop and maintain data infrastructure supporting real-time and batch processing for machine learning model training and inference.
• Collaborate with ML engineers and data scientists to optimize data access patterns and improve pipeline performance.

Requirements

• Master’s Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years experience OR Bachelor’s Degree AND 6+ years experience in business analytics, data science, software development, data modeling, or data engineering.
• Experience building and maintaining production data pipelines at scale using technologies such as Apache Spark or Kafka.
• Experience writing production-quality Python, Scala, or Java code for data processing applications.
• Experience with cloud data platforms (Azure, AWS, or GCP) and their data services.
• Experience with data orchestration frameworks such as Airflow.

Nice to have

• Experience building and scaling experimentation frameworks.
• Experience with schema management and data governance practices.
• Experience with real-time data processing and streaming architectures.
• Experience with containerization technologies (Docker, Kubernetes) for data pipeline deployment.
• Demonstrated experience with data quality frameworks and monitoring solutions.

Culture & Benefits

• Join a team that fosters a growth mindset, innovation, and collaboration to achieve shared goals.
• Work within a culture of inclusion built on values of respect, integrity, and accountability.
• Take ownership of the full data lifecycle for Copilot, impacting millions of users worldwide.
• Opportunity for freedom to innovate and support to build world-class data products.
• Eligible for benefits and other compensation; refer to Microsoft careers for details.
• The position will be open for a minimum of 5 days, with applications accepted on an ongoing basis.