Azure Databricks Technical Lead

Location: Anywhere in the UK. Hybrid working across company offices, client sites, and home; working from home 100% of the time is not possible.

Company

hirify.global, part of the Capgemini Group, delivers technology solutions and business transformation services for clients through local teams with global scale.

What you will do

• Implement, configure, troubleshoot, and optimise Azure Databricks platforms, including Serverless workloads, clusters, compute policies, autoscaling, and monitoring.
• Design and build data ingestion and transformation pipelines with full, incremental, and CDC loading patterns.
• Develop Databricks SQL and Delta Lake solutions and implement data-quality controls, schema evolution, reconciliation, and error handling.
• Integrate databases, SaaS applications, cloud storage, enterprise systems, and REST APIs using secure authentication and credential management.
• Support production operations through incident management, troubleshooting, performance tuning, governance, and platform standards.
• Collaborate with Data Engineers, Data Scientists, and Platform Teams while providing technical documentation and knowledge transfer.

Requirements

• Strong Python, PySpark, and SQL development skills.
• Hands-on Azure Databricks platform engineering and production-support experience.
• Experience with Azure Identity and Access Management, Networking, Private Endpoints, Security, Key Vault, Monitoring, and operational support.
• Experience with resource tagging, cost monitoring, budget controls, and usage optimisation.
• Experience with BI tooling integration, scheduling, application deployment in Databricks, and Agile delivery.
• Successful candidates must complete pre-employment checks covering identity, nationality or immigration status, three years of employment history, and an unspent criminal record check.

Nice to have

• POSIT/RStudio migration experience.
• AI/ML and LLM integration, RAG, and vector database exposure.
• Experience in regulated industries.

Culture & Benefits

• Hybrid-first working with flexible working arrangements available to UK employees.
• Training and development through courses, thinktanks, hackathons, and external certifications.
• Access to more than 250,000 learning courses, including AWS, Microsoft, Harvard Management, and cybersecurity qualifications.
• Inclusive recruitment under the UK Disability Confident Employer Level 2 scheme.
• Collaboration with diverse technology, engineering, and industry specialists.