Senior Python Data Engineer (Azure)

TL;DR

Senior Python Data Engineer (Azure): Designing and implementing ETL/ELT pipelines for investment data in a hybrid cloud environment with an accent on building Microsoft Fabric pipelines and working with Azure data services. Focus on optimizing SQL queries, managing ETL/ELT processes for structured and semi-structured data, and ensuring data governance.

Location: Onsite in Bengaluru

Company

hirify.global is a global software engineering firm that partners with a leading global investment management firm to provide IT solutions.

What you will do

• Develop, deploy, and maintain data pipelines using Azure Data Factory, Microsoft Fabric, and other Azure data services.
• Design, implement, and optimize SQL queries and stored procedures to ensure high performance and data accuracy.
• Write efficient, scalable Python code for data processing, transformation, and analysis.
• Build and maintain Spark and PySpark workflows for distributed, large-scale data processing.
• Organize and manage ETL/ELT processes for structured and semi-structured data.
• Implement data warehousing solutions and dimensional data models, ensuring data governance, metadata management, and security best practices.

Requirements

• 10+ years of experience in data engineering.
• Strong SQL skills (expert-level preferred).
• Strong hands-on experience with Python (Java is acceptable).
• Experience with Azure data services, especially Azure Data Factory or Microsoft Fabric.
• Practical experience with Spark (preferably PySpark).
• Knowledge of data warehousing concepts.
• Knowledge of data governance and data management.
• Practical development experience in cloud environments.

Nice to have

• Hands-on experience with MS SQL, PostgreSQL, or Amazon Redshift.
• Experience with Microsoft Fabric.
• Experience with AWS.
• Experience with Informatica or Pentaho.
• Experience with metadata management.
• Experience with Java or Scala.
• Experience with Power BI.

Culture & Benefits

• Vacation and sick pay as per country laws, including 10 days sick leave without a doctor's note.
• Health insurance for you and your loved ones.
• Time off for state holidays.
• Supportive, flexible, and innovative tech environment.
• We help cover the cost of IT certifications and provide access to top-tier courses and learning platforms.
• Comfort service for solving technical and everyday problems at work.
• Opportunities for corporate parties and small get-togethers.