Senior Azure Databricks Data Engineer (Azure)

TL;DR

Senior Azure Databricks Data Engineer (Azure): Designing and maintaining cloud-based ETL pipelines for Airline Crew Operations Data Analytics with an accent on PySpark and Azure Data Factory. Focus on developing scalable data-processing solutions and ensuring data quality through rigorous validation and reconciliation.

Location: Hybrid in Fort Worth, TX

Salary: $60 - $65 per hour

Company

hirify.global is a professional staffing and consulting firm providing specialized IT talent for large-scale enterprise projects.

What you will do

• Design, develop, and maintain ETL data pipelines using Azure Data Factory and Databricks.
• Build scalable data-processing solutions utilizing PySpark and Python.
• Execute data validation, quality checks, and reconciliation to ensure data reliability.
• Transform operational data for downstream analytics and reporting.
• Troubleshoot pipeline failures and resolve performance issues or data inconsistencies.
• Collaborate with business analysts and stakeholders in an Agile environment to define requirements.

Requirements

• 7+ years of overall experience in data engineering or software development.
• 3-5 years of hands-on experience with Databricks, PySpark, and Python.
• At least 3 years of experience with Microsoft Azure and Azure Data Factory.
• 3-5 years of experience in data validation and data-quality testing.
• Must be based in or able to work hybrid in Fort Worth, TX.
• 5+ years of experience working within an Agile development framework.

Nice to have

• Experience optimizing Databricks workloads and large-scale pipelines.
• Familiarity with Azure-based data architecture and deployment processes.

Culture & Benefits

• Access to medical, dental, and vision insurance.
• 401K contribution options.
• Paid time off (PTO) and sick leave.
• Work within a dynamic Agile environment focusing on aviation operations.