Lead Data Engineer (Databricks, Spark)

TL;DR

Lead Data Engineer (Databricks, Spark): Designing, building, and maintaining data pipelines and ETL processes with an accent on Lakehouse architecture and performance optimization. Focus on integrating cloud services and collaborating with data scientists to enable advanced analytics and machine learning workloads.

Location: Remote (Bogota, Colombia)

Company

AI-powered global business and technology transformation partner delivering end-to-end services and solutions across more than 50 countries.

What you will do

• Design and maintain scalable data pipelines and ETL processes using Databricks and Apache Spark.
• Implement and manage data Lakehouse architecture and ingestion from diverse sources.
• Optimize data workflows for maximum performance, scalability, and cost efficiency.
• Collaborate with data scientists and analysts to support advanced analytics and ML workloads.
• Ensure rigorous data quality, governance, and security across all organizational data assets.
• Integrate Databricks with major cloud providers (AWS, Azure, or GCP) and enterprise systems.

Requirements

• Hands-on professional experience with Databricks, Apache Spark, and PySpark.
• Advanced proficiency in SQL, Python, and data modeling principles.
• Experience with cloud platforms such as AWS, Azure, or GCP and their native data services.
• Familiarity with Delta Lake, Lakehouse architecture, and data governance frameworks.
• Understanding of CI/CD pipelines and DevOps practices specifically for data workflows.

Culture & Benefits

• Collaborative global community of colleagues providing support and inspiration.
• Empowerment to shape your own career path within the organization.
• Opportunity to work with the world's leading organizations to unlock technology value.
• Inclusive and diverse work environment with over 420,000 team members worldwide.