Data Engineer (PySpark)

TL;DR

Data Engineer (PySpark/AWS): Designing and building modern machine learning and AI services to support analytics and product improvement with an accent on data pipeline optimization and scalable cloud platforms. Focus on integrating diverse datasets, maintaining orchestration using Airflow, and supporting GenAI infrastructure.

Location: Flexible across all esure and hirify.global locations; must be based in the UK

Salary: £40,000 - £80,000

Company

One of the largest car and home insurers in the UK.

What you will do

• Build and support data products within a modern cloud data platform.
• Develop and optimize data pipelines and integrate data from varied sources with high quality standards.
• Maintain orchestration, monitoring, and performance of various data components.
• Collaborate with data scientists, architects, and product teams in multi-functional agile squads.
• Experiment with emerging data, ML, and AI technologies.
• Support infrastructure and workflows for ML and Generative AI.

Requirements

• Strong experience with Databricks or Snowflake on AWS.
• Proven proficiency in Python, including Spark and Airflow expertise.
• Advanced SQL skills and end-to-end data modelling experience.
• Hands-on CICD skills with Git, Jenkins, or similar tools.
• Knowledge of Terraform or similar Infrastructure as Code (IaC) tools.
• Must be based in the UK

Nice to have

• Experience with Docker and Kubernetes.
• Exposure to production Generative AI.

Culture & Benefits

• Flexible working options, including part-time and job share arrangements.
• Minimum of 35 days holiday (including bank holidays) with the ability to buy and sell days.
• Comprehensive health support including dental insurance, health cash plan, and mental health first aiders.
• Financial benefits such as annual bonus schemes, salary reviews, and competitive pensions.
• Generous parental leave entitlements for primary and non-pregnant parents.
• Discounts on gadgets, motor and home insurance, and cycle schemes.