Lead Expert - Data & AI Engineer (Databricks)

TL;DR

Lead Expert - Data & AI Engineer (Databricks/PySpark): Designing and maintaining robust data pipelines and managing initiatives for seamless data integration and transformation with an accent on scalable architecture and automated data quality. Focus on implementing efficient ingestion processes, optimizing CI/CD pipelines, and collaborating with data scientists to drive impactful science-led projects.

Location: Barcelona, Spain

Company

A global science-led company specializing in nutrition, health, and beauty, focusing on sustainable and innovative biotech breakthroughs.

What you will do

• Design and implement efficient pipeline architectures for data ingestion, transformation, and storage.
• Develop modular, efficient code for data integration following modern development practices.
• Perform ETL processes and data ingestion from source systems when tables are unavailable on the platform.
• Conduct regular pipeline maintenance and upgrades to ensure optimal performance.
• Implement automated data quality testing and validation to maintain high standards of integrity.
• Collaborate with data modelers, scientists, and BI developers to deliver impactful solutions.

Requirements

• 6+ years of hands-on experience in designing and building scalable data pipelines.
• Expert knowledge of dbt, SQL, Python, PySpark, Git, Azure DevOps, and Databricks.
• Strong experience in cloud engineering (AWS/Azure) with a focus on CI/CD pipeline development.
• Knowledge of data governance and FAIR principles.
• Proven ability in stakeholder engagement, project oversight, and continuous improvement.
• Resume must be submitted in English.

Culture & Benefits

• Unique career paths across the health, nutrition, and beauty sectors.
• Opportunity to work on cutting-edge research and biotech breakthroughs.
• Collaborative team environment with shared wins and professional development support.
• Focus on sustainability and creating products that impact millions of consumers daily.

Hiring process

• Online application via the company career portal with a resume in English.