Data Engineer

TL;DR

Data Engineer: Building and optimizing robust ETL/ELT pipelines and data infrastructure to empower analytical workloads with an accent on SparkSQL, Python, and dimensional modeling. Focus on implementing automated CI/CD workflows, ensuring high data quality, and optimizing performance within the Databricks Medallion architecture.

Location: Jakarta, Indonesia (Hybrid)

Company

hirify.global is a leading multi-line insurer that serves people and businesses in more than 200 countries and territories.

What you will do

• Design and implement ETL/ELT pipelines using Spark SQL and Python within Databricks Medallion architecture.
• Develop dimensional data models following star schema methodology, including SCD implementation and optimization.
• Optimize Spark SQL and DataFrame operations to maximize performance and minimize infrastructure costs.
• Build comprehensive data quality frameworks with automated validation, profiling, and reconciliation processes.
• Establish CI/CD pipelines incorporating version control and automated testing suites.
• Implement data governance standards, including row-level and column-level security policies.

Requirements

• 5+ years of experience with SparkSQL, Python, and PySpark for data engineering workflows.
• Strong proficiency in dimensional modeling and star schema design for analytical workloads.
• Experience implementing automated testing and CI/CD pipelines for data workflows.
• Familiarity with GitHub operations and collaborative development practices.
• Experience with cloud data services and infrastructure (AWS, Azure, or GCP).
• Proficiency with IDE tools such as Visual Studio Code.

Nice to have

• Experience with the Databricks platform.

Culture & Benefits

• Opportunity to work in a global, inclusive, and equal opportunity environment.
• Focus on innovation and challenging the status quo.
• Commitment to sustainability and protecting customers and the planet.
• Collaborative team culture that values unique employee contributions.