Lead Big Data Software Engineer (Databricks + AWS)

TL;DR

Lead Big Data Software Engineer (Databricks + AWS): Leading the design and development of scalable data platforms on AWS with an accent on the Databricks ecosystem and Lakehouse architectures. Focus on building high-performance batch and streaming data pipelines and guiding engineering teams through the full project lifecycle.

Location: Must be based in Poland or Ukraine (Remote/Office options available)

Company

Global IT services provider specializing in software development and digital transformation.

What you will do

• Design and implement scalable data solutions using Databricks on AWS, including Lakehouse architectures based on Delta Lake and Unity Catalog.
• Lead the development of batch and streaming data pipelines using Apache Spark or Flink.
• Define source-to-target mappings and drive architectural decisions for scaling the data platform and data models.
• Guide and support a team of data engineers, defining scope, priorities, and best practices.
• Manage workflows using orchestration tools such as Databricks Workflows, Apache Airflow, or MWAA.
• Collaborate with stakeholders to translate business requirements into technical implementation plans.

Requirements

• Proven experience as a Lead Data Engineer with a strong focus on data pipeline design.
• Strong expertise in AWS cloud platform and Databricks (Delta Lake, Unity Catalog, Workflows).
• Proficiency in Python, Scala, or Java, along with strong SQL skills.
• Practical experience with streaming platforms such as Apache Kafka, Amazon MSK, or Kinesis.
• Familiarity with data warehousing solutions such as Snowflake or Amazon Redshift.
• English: Upper-intermediate or higher level required.