Sr. Engineering Manager - Pipelines Engine (Data Infrastructure)

TL;DR

Sr. Engineering Manager (Data Infrastructure): Leading the Pipelines Engine team to build next-generation Runtime ETL features and Agentic Data Engineering infrastructure with an accent on performance and scalability. Focus on designing distributed data systems, managing technical architecture for exabyte-scale processing, and scaling top-tier engineering talent.

Location: Mountain View, California or San Francisco, California

Salary: $228,600 — $317,233 USD

Company

Databricks is a data and AI company providing a Data Intelligence Platform to unify data, analytics, and AI for over 10,000 organizations worldwide.

What you will do

• Lead the engineering team building next-generation ETL features for the Databricks Lakeflow platform.
• Spearhead Agentic Data Engineering infrastructure to power agentic pipeline authoring, execution, and maintenance.
• Oversee recruitment, hiring, and upskilling of top-tier engineering talent.
• Develop processes to implement product vision and strategy aligned with organizational goals and priorities.
• Manage long-term technical architecture decisions and balance the product roadmap against technical debt.
• Ensure the delivery of high-quality software that is reliable and easy to operate.

Requirements

• Minimum 3 years of experience in managing top-tier engineering teams.
• 5+ years of experience building data infrastructure systems (e.g., Apache Spark or database internals).
• Deep passion for and expertise in distributed systems, storage systems, or performance optimization.
• Proven experience in attracting, hiring, and coaching engineers to meet high hiring standards.
• Experience collaborating with product management and customers to translate business needs into technical requirements.
• Expertise in managing SLAs, testing, and quality for complex infrastructure services.

Culture & Benefits

• Comprehensive benefits and perks tailored to the specific region.
• Opportunity to work on one of the world's largest data engineering platforms processing exabytes of data daily.
• Inclusive culture committed to diversity, equal employment opportunity, and fostering an environment where everyone can excel.
• Direct impact on the development of foundational AI and data infrastructure.