Databricks Engineer (Azure)

TL;DR

Databricks Engineer (Azure): Designing, implementing, and optimizing large-scale data pipelines using Databricks, PySpark, and Azure services with an accent on scalability, reliability, and performance. Focus on developing ETL/ELT processes, ensuring data governance, and deploying cutting-edge data solutions in a hybrid environment.

Location: Flexible hybrid policy, primarily based in Mississauga, ON.

Salary: CAD $115k–$130k/year

Company

hirify.global is a global consulting firm that combines creativity and innovative technology to deliver industry-leading digital solutions for financial services and technology firms.

What you will do

• Build and maintain scalable ETL/ELT pipelines leveraging Databricks, PySpark/Spark, and SQL.
• Integrate data from diverse sources including Azure Blob Storage and ADLS.
• Collaborate with cross-functional teams to prepare data for dashboards and BI tools.
• Optimize Databricks workloads for cost efficiency and performance, and troubleshoot data pipelines.
• Implement and manage data security, access controls, and governance standards using Unity Catalog.
• Deploy Databricks jobs, notebooks, and configurations using Databricks Asset Bundles and GitLab.

Requirements

• Strong expertise in Databricks, including Delta Lake, Unity Catalog, and Lakehouse Architecture.
• Proficiency in Azure Cloud Services and solid understanding of Spark and PySpark for big data processing.
• Experience with relational databases.
• Knowledge of Databricks Asset Bundles and GitLab.
• Familiarity with streaming frameworks like Spark Streaming and developing real-time data solutions.

Nice to have

• Experience in the financial services or banking industry.
• Strong analytical skills and attention to detail.
• Ability to prioritize and multi-task effectively in a fast-paced environment.

Culture & Benefits

• Opportunity to work in a multinational organization with global offices.
• 15 days of paid annual leave plus an additional 10 days of personal leave.
• Comprehensive insurance plan including medical, dental, vision, life, and long-term disability.
• Flexible hybrid work policy.
• RRSP with employer’s contribution up to 4%.
• Access to higher education certification policy and free Udemy for Business courses.
• Coaching opportunities with experienced colleagues from FinLabs and Center of Excellences groups.
• Engage in cutting-edge projects at leading tier-one banks and financial institutions.
• Truly diverse, fun-loving, and global work culture.