Data Engineer

TL;DR

Data Engineer (ETL/ELT): Design and develop ETL/ELT workflows and data pipelines for batch and real-time processing. Focus on data governance, lineage tracking, and metadata management, including integration with Google Dataplex for centralized governance and data quality enforcement.

Location: Irving, TX

Salary: $55/hr

Company

hirify.global provides staffing services.

What you will do

• Design and develop ETL/ELT workflows and data pipelines for batch and real-time processing.
• Build and maintain data pipelines for reporting and downstream applications using open source frameworks and cloud technologies.
• Implement operational and analytical data stores leveraging Delta Lake and modern database concepts.
• Optimize data structures for performance and scalability across large datasets.
• Apply best practices for data governance, lineage tracking, and metadata management, including integration with Google Dataplex for centralized governance and data quality enforcement.
• Troubleshoot and resolve issues in data pipelines and ensure high availability and reliability.

Requirements

• Strong understanding of data structures, modeling, and lifecycle management.
• Hands-on experience designing and managing data pipelines (ETL/ELT Expertise).
• Advanced skills in distributed data processing and transformation with PySpark.
• Experience implementing open table formats for analytics with NetApp Iceberg.
• Knowledge of Hadoop Ecosystem components (HDFS, Hive).
• Experience with Cloud Platforms: GCP (BigQuery, Dataflow), Delta Lake, and Dataplex.
• Proficiency in Python, Spark, SQL.
• Strong experience with Apache Airflow, including authoring and maintaining DAGs for complex workflows.
• Understanding of relational and distributed systems (Database & Reporting Concepts).