1 месяц назад
Senior Data Engineer (Databricks)
Вакансия из Hirify Global, списка международных tech-компанийДля мэтча и отклика нужен Plus
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Senior Data Engineer (Databricks/PySpark): Building and validating large-scale data pipelines across enterprise data platforms and Lakehouse architectures with an accent on data quality, advanced SQL, and cloud data processing. Focus on source-to-target reconciliation, root cause analysis, performance optimization, and automated monitoring across Databricks, Hadoop, Hive, and cloud environments.
Location: Dublin, Ireland; permanent onsite role.
Company
Digital transformation and technology services provider delivering solutions from ideation through implementation across banking, insurance, retail, higher education, food, healthcare, and manufacturing.
What you will do
- Validate data pipelines across ingestion, transformation, and consumption layers.
- Perform source-to-target reconciliation, data profiling, and data quality checks.
- Develop and validate PySpark and Databricks pipelines for large-scale datasets.
- Work with Hadoop, Hive, Data Warehouses, Data Lakes, and Lakehouse architectures.
- Write advanced SQL for reconciliation, root cause analysis, business-rule validation, and query optimization.
- Investigate production incidents, manage data defects, and implement proactive monitoring and automated quality checks.
Requirements
- 6+ years of experience in Data Engineering, Data Quality Engineering, or Data Testing.
- Hands-on experience with Databricks and PySpark.
- Strong knowledge of Hadoop ecosystem components, including Hive, HDFS, and Spark.
- Advanced SQL skills, including complex joins, window functions, CTEs, aggregations, and optimization.
- Experience with Data Warehouses, Data Lakes, Lakehouse architectures, star schema, snowflake schema, and dimensional modeling.
- Experience with Azure, AWS, or GCP.
Nice to have
- Automated data testing frameworks.
- Data observability and monitoring tools.
- CI/CD implementation for data pipelines.
- Delta Lake, Unity Catalog, or similar technologies.
- Airflow, Kafka, or other Apache ecosystem tools.
Culture & Benefits
- Permanent employment in Dublin.
- Work within an agile, technology-focused digital services environment.
- Collaboration with Data Engineers, Architects, and Analysts.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →