Sr Designated Support Engineer (Apache Spark)

Location: San Francisco Bay Area, California, United States. The role includes approximately 20% travel and onsite collaboration during customer engagements.

Salary: $141,700–$250,800 USD annually, excluding potential bonus, equity, and benefits.

Company

Databricks develops a unified Data and AI Platform used by organizations worldwide to build and scale data applications, analytics, and AI solutions.

What you will do

• Provide advanced troubleshooting and root cause analysis for performance and reliability issues across Spark, SQL, Delta, Streaming, and Databricks Runtime features.
• Use Spark UI metrics, DAGs, event logs, Mosaic AI Model Service, and related tooling to diagnose complex production issues.
• Define monitoring requirements with R&D and NOC teams to identify performance issues early in customer environments.
• Build rapid proofs of concept and test, deploy, and monitor engineering solutions for customer challenges.
• Create playbooks and knowledge-base content for Spark, ML, and AI workflows, and train customer teams on tuning and debugging best practices.
• Act as a trusted technical advisor, advocate for customers, and collaborate with Field Engineering, Sales, Product, and Engineering teams.

Requirements

• 8–12 years of experience designing, building, and troubleshooting distributed computing applications.
• At least 4 years delivering production-scale Spark, ML, or AI solutions using Python, Java, or Scala.
• Hands-on expertise with data lakes, SQL databases, cloud data warehousing, and ETL tools such as Snowflake, Redshift, or BigQuery.
• Deep knowledge of Spark internals, Delta or Iceberg, JVM optimization, memory management, and AI ecosystems.
• Practical experience with AWS, Azure, or GCP, plus CI/CD pipelines, monitoring, and alerting systems.
• 3–5 years of customer-facing experience in roles such as Technical Account Manager or Solutions Architect, with strong communication and cross-functional leadership skills.

Culture & Benefits

• Comprehensive employee benefits and region-specific perks.
• Inclusive hiring practices and a commitment to diversity and equal employment opportunity.
• Collaboration with global customers and cross-functional teams.