Senior Designated Support Engineer (Apache Spark)

TL;DR

Senior Designated Support Engineer (Apache Spark): Managing large strategic accounts and resolving complex big data performance bottlenecks with an accent on Apache Spark internals and Data Engineering best practices. Focus on optimizing ETL/ELT pipelines, troubleshooting distributed computing issues, and collaborating with engineering teams to resolve critical customer cases.

Location: Amsterdam, Netherlands

Company

Databricks is a leading data and AI company and the creator of the Lakehouse architecture, Apache Spark, Delta Lake, and MLflow.

What you will do

• Manage large strategic accounts to provide a high-touch, proactive support experience.
• Deep-dive into complex Apache Spark and Data Engineering issues using Spark UI, thread dumps, and logs.
• Advise customers on Data Engineering best practices, ETL/ELT optimization, and Lakehouse architecture.
• Coordinate with Engineering and Product teams to expedite the resolution of critical support tickets.
• Conduct regular case reviews to identify trends and proactive optimization opportunities.
• Champion customer needs and drive self-sufficiency through guidance on support tools.

Requirements

• Minimum 3 years of experience in Data Engineering, building and maintaining large-scale data pipelines.
• Minimum 3 years of experience designing and sustaining Python, Java, or Scala-based applications.
• Deep architectural knowledge of Apache Spark internals (Drivers, Executors, DAGs, Catalyst, Tungsten).
• Proven track record of troubleshooting Spark issues such as data skew and OOM errors.
• Strong analytical skills applied to distributed big data computing environments.
• Excellent communication skills for translating complex technical ideas to non-technical stakeholders.

Culture & Benefits

• Comprehensive benefits and perks tailored to the employee's region.
• Inclusive culture committed to diversity and equal employment opportunity.
• Opportunity to work with a platform relied upon by over 50% of the Fortune 500.