Senior Data Engineer (Hadoop)

TL;DR

Senior Data Engineer (Hadoop): Designing and developing custom data pipelines for a banking project with an accent on data ingestion from diverse sources and performance optimization over HDFS. Focus on debugging Spark/Impala sessions, designing secure data solutions, and contributing to the long-term data strategy of the platform.

Location: Hybrid in Prague, Czechia

Salary: 85 000 – 90 000 CZK per year

Company

hirify.global is an IT consultancy firm that helps startups, mid-sized and large corporations to create and deliver innovative technologies, specializing in AI, ML, BI, Web, and Automation solutions for various sectors.

What you will do

• Design and develop custom data pipelines for batch and near-real-time processing.
• Ingest data from diverse sources and debug platform/application errors.
• Optimize performance over Avro and Parquet on HDFS, including partitioning and bucketing.
• Design secure data solutions following security-by-design principles.
• Monitor Big Data applications using Kafka, InfluxDB, and Grafana.
• Perform DevOps tasks, including CI/CD with Jenkins, Git, and SVN.
• Mentor junior colleagues and contribute to the platform's long-term data strategy.

Requirements

• Strong experience with the Cloudera Hadoop stack: HDFS, Hive Metastore, Impala, Spark.
• Advanced SQL skills and scripting experience in Bash, Python, and PowerShell.
• Experience in Linux production environments.
• Familiarity with Airflow, Informatica, and NiFi.
• Proven ability to independently design complex data solutions.

Nice to have

• Experience with Databricks – Data Engineering.
• Exposure to Azure Data Stack (Data Factory, ADLS, Synapse, etc.).

Culture & Benefits

• Collaborate with a team that has a successful track record in delivering innovative technologies.
• Work on challenging projects for banking sector clients.
• Opportunity to mentor junior colleagues and transfer technical know-how.