Senior Big Data Engineer (Databricks + AWS)

TL;DR

Senior Big Data Engineer (Databricks + AWS): Developing scalable cloud-based data solutions on AWS with an accent on batch and streaming data pipelines. Focus on optimizing data processing with PySpark and SQL, implementing Delta Lake architectures, and managing complex workflows via Airflow.

Location: Poland (Remote/Office)

Company

hirify.global is a global IT services provider specializing in engineering and technology solutions.

What you will do

• Design, develop, and maintain scalable batch and streaming data pipelines.
• Build and optimize data processing solutions using Python (PySpark) and SQL.
• Develop and manage data workflows using Databricks on AWS.
• Implement and maintain Delta Lake-based data architectures.
• Utilize orchestration tools such as Apache Airflow or MWAA to manage workflows.
• Support streaming solutions using Apache Kafka, Amazon MSK, or Kinesis.

Requirements

• 4+ years of experience in Big Data or Data Engineering.
• Strong proficiency in Python (PySpark) and SQL.
• Hands-on experience with AWS cloud services and Databricks on AWS.
• Strong knowledge of Apache Spark and large-scale data processing.
• Experience with Delta Lake, Airflow, and streaming technologies (Kafka, MSK, or Kinesis).
• English: Upper-intermediate (B2) or higher required

Culture & Benefits

• Equal opportunity employer regardless of race, color, religion, or gender identity.
• Opportunity to work on the full project lifecycle from PoC and MVP to production.
• Collaboration with cross-functional technical and business stakeholders.