Senior Data Engineer

TL;DR

Senior Data Engineer: Responsible for designing and building scalable data pipelines using Azure Data Factory and Azure Synapse, focusing on ingesting, processing, and transforming high-volume drilling data. Focus on data quality, availability, consistency by implementing automated validation checks and optimizing data pipelines for performance and scalability.

Location: Hybrid in Costa Rica

Company

Encora is a global company that offers Software and Digital Engineering solutions.

What you will do

• Design and build scalable data pipelines to ingest, process, and transform high volume drilling data using Azure Data Factory and Azure Synapse.
• Implement complex logic to convert time-based drilling sensor data into depth-based datasets.
• Develop ingestion frameworks to process diverse drilling data formats from multiple vendors.
• Ensure data quality, availability, and consistency by implementing automated validation checks and optimizing data pipelines.

Requirements

• Strong experience designing and implementing data pipelines using Azure Data Factory and Azure Synapse Analytics.
• Advanced proficiency in SQL and strong programming skills in Python with experience using PySpark or Pandas for data processing.
• Experience working with big data technologies such as Spark and managing data in Azure Data Lake Storage Gen2.
• Knowledge of common data formats such as JSON, Parquet, Delta Lake, and XML.
• Understanding of ETL and ELT patterns and modern data engineering best practices.

Nice to have

• Microsoft Certified Azure Data Engineer Associate (DP 203) certification.

Culture & Benefits

• Encora hires professionals based solely on their skills and does not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.