Lead Big Data Engineer (Databricks + AWS)

TL;DR

Lead Big Data Engineer (Databricks + AWS): Designing and developing scalable data platforms on AWS with an accent on Lakehouse architectures and real-time data processing. Focus on building optimized batch and streaming pipelines, defining architectural best practices, and mentoring engineering teams.

Location: Poland (Remote/Office)

Company

hirify.global is a global digital transformation company providing comprehensive software development and data analytics services.

What you will do

• Design and optimize scalable data solutions on AWS using Databricks, Delta Lake, and Unity Catalog.
• Develop batch and real-time processing solutions with Apache Spark, Flink, Kafka, and Kinesis.
• Lead data integration and migration activities, including source-to-target mapping across multiple data sources.
• Drive data platform architecture and engineering best practices to ensure scalability and maintainability.
• Collaborate with business and technical stakeholders to translate requirements into implementation roadmaps.
• Mentor data engineering teams and manage automated workflows via Airflow, MWAA, and Databricks Workflows.

Requirements

• Proven experience as a Lead Data Engineer delivering scalable data platforms and pipelines.
• Hands-on expertise with Apache Spark, Flink, Kafka, Amazon MSK, or Kinesis.
• Strong proficiency in AWS and Databricks (Delta Lake, Unity Catalog, Workflows).
• Advanced SQL skills combined with proficiency in Python, Scala, or Java.
• Experience building and orchestrating data workflows using Airflow, MWAA, or Databricks Workflows.
• English: Upper-intermediate (B2) or higher required.

Nice to have

• Practical knowledge of Snowflake and Amazon Redshift.
• Familiarity with Avro, GitHub, and SQL-based version control systems.

Culture & Benefits

• Collaborative and innovation-driven engineering environment.
• Opportunity to work with cutting-edge technologies in the Databricks ecosystem.
• Active knowledge sharing within the professional engineering community.
• Commitment to diversity and equal opportunity employment.