Spark / Databricks Platform Engineer (Databricks)

TL;DR

Spark / Databricks Platform Engineer (Databricks): Designing, developing, and optimizing scalable data solutions on the Databricks platform with an accent on ingestion pipelines, medallion architecture, and dimensional data modeling. Focus on implementing Lakeflow Declarative Pipelines, managing Delta Lake structures, and delivering operational insights via Databricks SQL dashboards.

Location: Hybrid - Austin, TX (US)

Company

A technology and professional services firm specializing in innovative technologies since 2004.

What you will do

• Design and optimize scalable data solutions using Apache Spark, PySpark, or Scala for large-scale processing.
• Build and maintain robust ETL/ELT ingestion pipelines utilizing Lakeflow Declarative Pipelines.
• Implement Delta Lake architectures and medallion database structures across Bronze, Silver, and Gold layers.
• Orchestrate and monitor production jobs using Lakeflow Jobs or Apache Airflow.
• Develop enterprise data warehouses utilizing dimensional data modeling, including Star and Snowflake schemas.
• Create dynamic analytical dashboards and reporting solutions natively within Databricks SQL and Databricks Apps.

Requirements

• 8+ years of experience in IT, specifically designing and delivering technology solutions.
• 8+ years of experience with Databricks, Apache Spark, and building ETL/ELT data pipelines.
• 8+ years of experience in data warehousing and dimensional data modeling.
• Professional proficiency in SQL and Python or Scala for large-scale data processing.
• Experience implementing data governance, security controls, and data quality metrics.
• Must be based in or able to work in Austin, TX (Hybrid setup).

Nice to have

• Experience working within public sector or state government environments.
• Implementation of CI/CD practices for data pipelines, including DevOps and Git-based workflows.
• Professional Databricks certification (Associate or Professional).

Culture & Benefits

• Competitive salary.
• Flexible work-from-home options available.
• Exposure to cutting-edge data platform technologies.
• Collaborative environment working with cross-functional stakeholders and technical peers.