Senior Data Engineer (Databricks)

TL;DR

Senior Data Engineer (Databricks/AWS): Designing, developing, and optimizing high-performance data pipelines and workflows leveraging Databricks and AWS with an accent on production troubleshooting and data modeling. Focus on implementing CI/CD workflows, improving pipeline reliability through monitoring and logging, and optimizing Spark-based transformations.

Location: Hybrid in London, UK

Company

hirify.global is a leading global provider of asset servicing and operational solutions for public and private funds, operating as a division of Mitsubishi UFJ Financial Group.

What you will do

• Design and maintain high-performance data pipelines using Databricks.
• Provide L2 and L3 support to troubleshoot and rectify production pipeline issues.
• Collaborate with architects and analysts to build robust data models and transformations.
• Implement monitoring and logging strategies to ensure pipeline reliability and performance.
• Deploy and manage code through CI/CD workflows.
• Ensure data quality, security, and compliance within Databricks environments.

Requirements

• Proficiency in Databricks, including PySpark, Spark SQL, and Delta Lake.
• Strong experience in data modeling, optimization, and performance tuning.
• Hands-on experience with AWS infrastructure, specifically AWS DMS and S3-based ingestion patterns.
• Solid understanding of data governance, security best practices, and compliance.
• Familiarity with CI/CD principles and practices.
• Proven experience supporting and troubleshooting data pipelines.

Nice to have

• Databricks Certified Data Engineer Professional.
• Prior experience working within financial services or highly regulated environments.

Culture & Benefits

• Hybrid working arrangement.
• Competitive compensation package.
• Significant professional growth and learning opportunities.
• Collaborative and innovative team culture.