Senior Data Engineer (Azure)

TL;DR

Senior Data Engineer (Azure): Building and optimizing scalable ETL/ELT pipelines for complex sensor data with an accent on Azure Synapse, PySpark, and real-time stream processing. Focus on standardizing multi-vendor oilfield data and ensuring high-performance data ingestion frameworks within an Azure Data Lake environment.

Location: Must be based in Brazil

Company

hirify.global is a global digital engineering and modernization partner, providing expertise in product development, cloud services, and data analytics for leading enterprises worldwide.

What you will do

• Architect and build scalable ETL/ELT pipelines using Azure Data Factory and Synapse.
• Convert complex time-indexed sensor data into depth-indexed formats.
• Create ingestion frameworks for oilfield file formats like LAS, DLIS, and WITSML.
• Standardize sporadic vendor data into common schemas and implement automated quality checks.
• Tune Spark jobs and SQL queries for performance and cost efficiency.
• Manage Azure Data Lake Storage (ADLS Gen2) architecture and implement Row-Level Security.

Requirements

• Location: Must be based in Brazil.
• Strong experience designing data pipelines with Azure Data Factory and Synapse Analytics.
• Advanced SQL skills and experience with large-scale data processing.
• Proficiency in Python using PySpark or Pandas.
• Experience with distributed data processing using Spark and data formats like Parquet and Delta Lake.
• Knowledge of ETL/ELT patterns and experience handling time-series or streaming data.

Nice to have

• Microsoft Certified Azure Data Engineer Associate (DP 203) certification.

Culture & Benefits

• Professional growth within a global team of over 9,000 experts.
• Exposure to diverse digital engineering projects across various domains.
• Focus on skill-based hiring and an inclusive, non-discriminatory work environment.
• Full-time engagement with remote work flexibility.