Principal Data Engineer (Databricks/Snowflake)

TL;DR

Principal Data Engineer (Databricks/Snowflake): Designing and building greenfield cloud-native data platforms and real-time pipelines with an accent on data mesh, data products, and data contracts. Focus on optimizing high-performance streaming architectures, implementing scalable storage patterns, and providing technical leadership for complex data ecosystems.

Location: Hybrid in London, United Kingdom

Company

A global, independent technology consultancy that designs and builds modern data platforms and intelligent systems at the intersection of data engineering, software engineering, and AI.

What you will do

• Lead the end-to-end architecture of modern, cloud-native data platforms using data mesh and data contracts.
• Design and deliver scalable data ecosystems utilizing Databricks, Snowflake, AWS, and GCP.
• Build and optimize batch and real-time pipelines using Kafka, Flink, Kinesis, and Airflow.
• Manage large-scale data transformations and storage across relational, NoSQL, and analytical stores like Postgres, BigQuery, and Cassandra.
• Implement secure, compliant data solutions with security by design and integrated governance.
• Consult directly with clients to translate complex business needs into pragmatic engineering decisions.

Requirements

• Strong proficiency in Python and SQL.
• Deep experience with Spark and modern data platforms (Databricks or Snowflake).
• Demonstrated ownership of large-scale data platform architectures and strong data modelling skills.
• Solid experience with cloud data services on AWS or GCP.
• Proficiency with Infrastructure-as-Code (Terraform, Pulumi) and CI/CD pipelines (GitHub Actions, ArgoCD).
• Experience working within consulting or professional services environments.

Nice to have

• Experience with data testing and quality frameworks such as dbt, Great Expectations, or Soda.

Culture & Benefits

• Work on high-impact, greenfield projects rather than maintaining legacy systems.
• Collaboration with a high-calibre team of senior engineers.
• Significant autonomy, technical influence, and opportunities for professional growth.
• Focus on engineering excellence and delivering real-world outcomes.