Senior Data Engineering (Azure)

TL;DR

Senior Data Engineering (Azure/Spark): Designing and developing robust, scalable data pipelines to transform and process large datasets with an accent on performance optimization and data governance. Focus on optimizing high-throughput workflows and collaborating with stakeholders to translate business needs into technical specifications.

Location: Remote (Brazil)

Company

Tech transformation specialists uniting human expertise with AI to create scalable tech solutions for over 1,000 clients worldwide.

What you will do

• Design and develop scalable data pipelines using Databricks, Apache Spark, and SQL to process large datasets efficiently.
• Monitor and optimize pipeline performance to ensure high throughput and low latency.
• Collaborate with data scientists, analysts, and business stakeholders to translate requirements into technical specifications.
• Implement data quality checks and governance practices to ensure consistency, accuracy, and compliance.
• Maintain comprehensive documentation and contribute to team best practices.

Requirements

• Solid experience in data engineering with a specific focus on data pipelines.
• Proficiency in Databricks, Apache Spark, SQL, and Python or Scala.
• Hands-on experience with Azure cloud technologies.
• Excellent communication skills in both English and Portuguese.
• Bachelor’s degree in Computer Science, Information Technology, or a related field.

Culture & Benefits

• Comprehensive health and dental insurance.
• Meal and food allowances and childcare assistance.
• Profit Sharing and Results Participation (PLR).
• Access to continuous learning via hirify.global University and other online platforms.
• Wellness partnerships with Wellhub (Gympass) and TotalPass.
• Extended paternity leave and life insurance.