Senior Data Engineer (Databricks)

TL;DR

Senior Data Engineer (Python/Spark): Building and optimizing data ingestion pipelines and transformations based on the Databricks Platform with an accent on data discovery and data lineage. Focus on writing robust code for complex data transformations and collaborating across different time zones.

Location: Remote or office work, globally available. Collaboration with US time zones (EST or PST) is required.

Company

hirify.global is an international team with colleagues working from offices and remotely worldwide.

What you will do

• Design and create data ingestion pipelines based on the Databricks Platform.
• Prepare data transformations using Python and PySpark.
• Conduct Data Discovery with Data lineage to build Data Domains.

Requirements

• 5+ years of experience in Data Engineering.
• Strong familiarity with the Databricks platform (including Unity Catalog).
• Ability to write robust code with Python.
• Proficiency with Spark (PySpark).
• Knowledge of cloud platforms, specifically AWS.
• Ability to collaborate closely with US time zones (EST or PST).

Nice to have

• Knowledge of Lakehouse concept, including experience using Delta Lake optimized storage layer.

Culture & Benefits

• Competitive compensation.
• Flexible working hours.
• Healthcare benefits: medical insurance and paid sick leave.
• Continuous education, mentoring, and professional development programs.
• Opportunity for company-paid certifications.
• Work with a team of excellent tech expertise.