Data Engineer (Databricks)

TL;DR

Data Engineer (Databricks): Designing, building, and optimizing large-scale data solutions on the Databricks platform with an accent on data quality, governance, and security. Focus on leading the development of scalable data pipelines and collaborating with cross-functional teams to deliver actionable insights.

Company/Компания

What you will do/Что делать

• Lead the design, development, and deployment of scalable data pipelines and ETL processes using Databricks (Spark, Delta Lake, MLflow);
• Architect and implement data lakehouse solutions, ensuring data quality, governance, and security;
• Optimize data workflows for performance and cost efficiency on Databricks and cloud platforms (Azure, AWS, or GCP);
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver actionable insights;
• Mentor and guide junior engineers, promoting best practices in data engineering and Databricks usage;
• Monitor, troubleshoot, and resolve issues in production data pipelines and environments;

Requirements/Требования

• Bachelor’s or Master’s degree in Computer Science, Engineering, Mathematics, or a related field;
• 5+ years of experience in data engineering, with at least 2 years of hands-on experience with Databricks (including Spark, Delta Lake, and MLflow);
• Strong proficiency in Python and/or Scala for data processing;
• Deep understanding of distributed data processing, data warehousing, and ETL concepts;
• Experience with cloud data platforms (Azure Data Lake, AWS S3, or Google Cloud Storage);
• Excellent problem-solving, communication, and leadership skills;
• English: Upper Intermediate level or higher;