Data Engineer (Big Data)

TL;DR

Data Engineer (Big Data/Python): Developing and optimizing big data pipelines and infrastructure with an accent on PySpark and SQL. Focus on implementing CI/CD processes using Jenkins, XL Deploy, and XL Release to ensure scalable data processing and deployment.

Location: Hybrid (2 days/week in Porto). Must be based in Portugal with Portuguese/EU citizenship or a valid work permit.

Company

Consulting firm specializing in matching tech talent with a diverse network of national and international partners across the tech industry.

What you will do

• Build and optimize big data pipelines using Python and PySpark.
• Apply data modeling principles and SQL for efficient data management and analysis.
• Manage source code and versioning utilizing Git.
• Implement and maintain deployment pipelines with Jenkins, XL Deploy, and XL Release.
• Collaborate within cross-functional and distributed teams to share knowledge and drive continuous improvement.

Requirements

• 4+ years of experience as a Data Engineer specializing in Big Data.
• Strong proficiency in Python and PySpark.
• Solid knowledge of SQL and Data Modeling principles.
• Hands-on experience with Jenkins, XL Deploy, and XL Release.
• English: B2 level fluency required.
• Must be based in Portugal with Portuguese or EU citizenship, or a valid work permit.

Culture & Benefits

• Health insurance and Coverflex meal card.
• Opportunities for career progression and personalized support from a career coach.
• Welcome kit and access to various special discounts.
• Strong focus on professional well-being and work-life balance.