Lead Data Engineer (AI)

TL;DR

Lead Data Engineer (AI): Building and optimizing a data platform on GCP for population-scale health data for pharma and biotech customers with an accent on designing ingestion patterns for huge data drops, cost, and observability. Focus on translating real-world medical questions into production systems and exploring cutting-edge AI and LLMs.

Location: London. Full UK working rights required; visa sponsorship is not available.

Company

A scaling Techbio company focused on handling sensitive, population-scale health data for pharma and biotech customers.

What you will do

• Take ownership of a GCP-based data platform.
• Design ingestion patterns for multi-terabyte datasets from third parties and Azure into BigQuery.
• Focus on cost optimization and observability for the data environment.
• Collaborate with internal medical teams to translate real-world questions into production systems.
• Work on cutting-edge AI, custom-build LLMs, and infrastructure.

Requirements

• Deep, hands-on experience running data platforms on GCP.
• Day-to-day use of Python.
• Experience operating BigQuery at scale and understanding query behavior across hundreds of terabytes.
• Built production systems with Airflow/Composer, DBT, Terraform, and Docker.
• Comfortable with GKE, Cloud Run, or Cloud Functions.
• Worked with streaming systems built on Dataflow/Apache Beam.
• Experience in regulated environments and alongside security teams, ideally with exposure to ISO27001.
• Ability to work closely with non-technical stakeholders and shape solutions.
• Full UK working rights required.

Culture & Benefits

• Work with a super smart, humble team.
• Opportunity for real ownership and freedom in your work.
• Chance to work on cutting-edge AI and custom-build LLMs.