Senior Data Developer (Databricks)

TL;DR

Senior Data Developer (Databricks): Building and optimizing a mission-critical data platform using medallion architecture with an accent on PySpark, Delta Lake, and dimensional modeling. Focus on designing scalable data pipelines, managing environment deployments, and serving as the primary technical reference for the platform.

Location: Remote (Colombia)

Company

hirify.global is a global technology company that helps large enterprises transform AI into real business impact through AI-native execution and tech-integrated solutions.

What you will do

• Own and evolve notebooks across the medallion architecture (Stage, Bronze, Silver, and Gold layers).
• Design and implement dimensional modeling artifacts, including facts, dimensions, and slowly changing dimensions (SCD).
• Manage data processing jobs and promote changes across Dev, QA, UAT, and PROD environments.
• Review pull requests to enforce code quality, performance, and architectural consistency.
• Monitor and troubleshoot production jobs using platform-native diagnostic tools and logs.
• Maintain the automated testing pipeline, CI/CD workflows, and technical documentation.

Requirements

• Proven production experience on the Databricks platform.
• Strong proficiency in PySpark (DataFrame API, Spark SQL) and Databricks SQL with performance tuning expertise.
• Practical experience with Delta Lake (MERGE/upsert, ACID transactions, time travel, ZORDER).
• Experience implementing Slowly Changing Dimensions (Type 1 and 2) and star schema design.
• Proficiency with Git, Azure DevOps, and Azure services (Key Vault, Data Lake Storage).
• English: C1 or above required for direct communication with US-based stakeholders.
• Must be based in Colombia.

Nice to have

• Experience with Databricks Asset Bundles or Infrastructure-as-Code (IaC).
• Familiarity with Delta Live Tables and Databricks Genie (AI/BI Genie).
• Experience with data quality frameworks like Great Expectations or Soda Core.
• Knowledge of pytest and databricks-connect for automated testing.
• Experience using AI-assisted development tools such as GitHub Copilot or Cursor.