Lead Data Engineer

TL;DR

Lead Data Engineer: Building and operating robust data pipelines on AWS with an accent on Lakehouse architectures and complex ETL/ELT orchestration. Focus on implementing Apache Iceberg, optimizing data processing workflows, and providing technical leadership to the engineering team.

Location: Must be based in Porto, Portugal (Hybrid)

Company

hirify.global is a specialized IT consulting partner with 18 years of experience, dedicated to delivering agile, people-centered solutions in a fast-moving market.

What you will do

• Build and operate scalable data pipelines within AWS production environments.
• Design and manage complex ETL/ELT workflows using Airflow and AWS Step Functions.
• Implement and optimize Lakehouse architectures using Apache Iceberg.
• Lead and mentor a team of engineers while driving technical standards.
• Develop automation strategies using CI/CD practices and infrastructure as code.
• Ensure data quality and performance through table compaction and Z-Order optimizations.

Requirements

• 8 to 10 years of experience in data engineering or related fields.
• At least one previous experience in a technical leadership capacity.
• Advanced proficiency in Python and SQL.
• Strong expertise with AWS services including Glue, Lambda, and DynamoDB.
• Deep understanding of PySpark and the Spark ecosystem.
• Experience with DevOps practices and infrastructure as code tools like Terraform or CloudFormation.

Nice to have

• Experience with Docker and Linux-based infrastructure.
• Knowledge of Azure or GCP as secondary cloud platforms.
• Proficiency in French.

Culture & Benefits

• No-term full-time contract with health insurance coverage.
• 22 days of paid vacation plus 4 extra days for holidays and personal time.
• Meal allowance provided via Coverflex.
• Annual budget for professional training and development.
• Referral bonus program and flexible company discounts.
• Collaborative, team-oriented culture within a growing international organization.