Data Engineer II (AI)

TL;DR

Data Engineer II (Python/Spark/AWS): Building and maintaining scalable data pipelines and services for the Search and AI Platform with an accent on ELT/ETL processes and cloud infrastructure. Focus on processing large-scale datasets, integrating search and graph data stores, and implementing CI/CD best practices.

Location: London Wall, London

Company

A global leader in information and analytics that helps researchers and healthcare professionals advance science and improve health outcomes.

What you will do

• Build and maintain data pipelines and services using Python, SQL, Spark, and Airflow.
• Contribute to ELT and ETL pipelines for data ingestion into search, graph, and relational data stores.
• Develop and manage cloud infrastructure on AWS.
• Write clean, well-tested code following best-practice software engineering and CI/CD principles.
• Utilize AI-assisted development tools such as Claude Code, Copilot, or Codex in the daily workflow.
• Collaborate with product, platform, and data science teams to deliver business value.

Requirements

• 2–3 years of commercial experience as a data or software engineer, including exposure to production systems.
• Proficiency in Python, SQL, and Spark.
• Basic understanding of batch processing and relational databases.
• Hands-on experience with data pipeline tools (Spark or Airflow is a plus).
• Familiarity with cloud platforms, preferably AWS.
• Collaborative mindset and strong communication skills.

Nice to have

• Exposure to data lake formats such as Delta Lake or Iceberg.
• Experience with search, graph, or large-scale analytical databases.
• Interest in AI, machine learning pipelines, or embedding generation.

Culture & Benefits

• Flexible working hours to support work/life balance.
• Comprehensive pension plan and generous vacation entitlement.
• Option for sabbatical leave and various family care leaves (Maternity, Paternity, Adoption).
• Personal Choice budget and internal employee discounts.
• Global Employee Assistance Program.