Senior Data Engineer (Spark)

TL;DR

Senior Data Engineer (Spark): Building and maintaining high-performance real-time data streaming platforms with an accent on ETL workflow optimization and cloud infrastructure management. Focus on designing scalable systems using Java, Python, and Node.js while ensuring reliable performance through advanced SQL querying and CI/CD automation.

Location: Hybrid (up to 80% remote, requires regular office attendance for collaboration).

Company

hirify.global is a leading international betting and gaming company, encompassing brands such as William Hill, 888, and Mr Green.

What you will do

• Design, build, and maintain real-time data pipelines using Apache Flink and Spark.
• Develop and optimize ETL workflows to process large-scale data volumes.
• Write efficient code in Java, Python, and Node.js for data processing tasks.
• Manage cloud-based data infrastructure primarily on AWS.
• Implement and maintain CI/CD pipelines for automated application deployment.
• Monitor and troubleshoot real-time data pipelines to ensure optimal performance.

Requirements

• 5+ years of experience in data engineering with a focus on real-time streaming platforms.
• Strong programming proficiency in Java, Python, and Node.js.
• Expertise in SQL for data manipulation and performance optimization.
• Solid experience with ETL tools and cloud platforms, specifically AWS.
• Familiarity with data storage solutions like Kafka, Hadoop, NoSQL, and relational databases.
• Understanding of containerization tools like Docker and Kubernetes.

Nice to have

• Experience with Apache NiFi or Airflow.
• Knowledge of Azure or Google Cloud platforms.

Culture & Benefits

• Competitive salary package.
• Hybrid working model with 20% office time for team collaboration.
• 22 days annual leave plus bank holidays and an extra birthday day.
• Comprehensive health and wellbeing support, including mental health resources and gym discounts.
• Industry-leading maternity/paternity leave and family support policies.
• Access to ongoing professional learning and development.

Hiring process

• Initial screening call.
• Interview stage.