Staff Designated Support Engineer (Big Data)

TL;DR

Staff Designated Support Engineer (Big Data): Deliver high-touch specialized support and tailored technical solutions for large strategic customers using Apache Spark and data technologies with an accent on troubleshooting, performance tuning, and AI/ML runtime capabilities. Focus on designing and deploying scalable Spark/ML/AI solutions, root cause analysis, and collaborating cross-functionally to solve complex production issues.

Location: San Francisco Bay Area, CA office with 50% travel to client location in San Francisco

Salary: $141,700–$250,800 USD

Company

Databricks is a leading data and AI company, creators of Apache Spark, Delta Lake, and MLflow, serving over 10,000 organizations worldwide including more than 50% of the Fortune 500.

What you will do

• Perform advanced troubleshooting and root cause analysis for Spark, SQL, Delta, Streaming, and Databricks runtime features.
• Collaborate with R&D and NOC teams to optimize customer environments and implement continuous monitoring.
• Build rapid POCs and deploy solutions showcasing advanced Spark, ML, and AI capabilities aligned with customer goals.
• Develop and maintain playbooks and knowledge bases for common issues and solutions.
• Train customer engineering and business teams on best practices in performance tuning and debugging.
• Advocate for customers and collaborate with Field Engineering, Sales, and Product teams during engagements.

Requirements

• Must have 8–12 years experience in distributed computing and 4+ years with production-scale Spark/ML/AI solutions using Python, Java, or Scala.
• Hands-on expertise with Data Lakes, SQL databases, and cloud data warehousing tools like Snowflake, Redshift, BigQuery.
• Deep knowledge of Spark internals, JVM optimization, and AI ecosystems including Machine Learning and Generative AI.
• Experience with AWS, Azure, or GCP and building/managing CI/CD pipelines and monitoring systems.
• 3–5 years customer-facing experience with strong communication and problem-solving skills.
• Proven ability to lead cross-functional teams and drive customer success.

Culture & Benefits

• Comprehensive benefits and perks tailored to employee needs.
• Commitment to diversity and inclusion ensuring equal employment opportunity.
• Global presence with headquarters in San Francisco and offices worldwide.