Эта вакансия в архиве
Посмотреть похожие вакансии ↓обновлено 10 дней назад
Middle Data Engineer (Azure Databricks)
Описание вакансии
Текст:
TL;DR
Middle Data Engineer (Azure Databricks): Designing and developing modern data pipelines and Lakehouse architectures using Azure Databricks and Spark with an accent on scalability and cloud-based data architectures. Focus on building optimized data transformations with PySpark, implementing Delta Lake architectures, and integrating legacy SQL Server environments into a cloud platform.
Location: Remote (Work from Anywhere) or Larnaca, Cyprus
Company
A global IT services and consulting company supporting digital transformation for large enterprises across 5 continents.
What you will do
- Design, develop, and maintain data pipelines using Azure Databricks.
- Build and optimize data transformations using PySpark and SQL in Databricks.
- Implement and maintain Lakehouse architectures using Delta Lake.
- Develop ETL/ELT pipelines orchestrated through Azure Data Factory.
- Integrate data from multiple sources and maintain data models for analytics.
- Collaborate with BI teams to support Power BI and reporting platforms.
Requirements
- 3+ years of experience in Data Engineering or Data Warehouse development.
- Hands-on experience with Azure Databricks and building pipelines using PySpark and Spark SQL.
- Solid understanding of distributed data processing, big data concepts, and Delta Lake.
- Strong SQL skills and experience with SQL Server relational databases.
- Experience building data pipelines using Azure Data Factory.
- Ability to handle large datasets and perform performance optimization.
Nice to have
- Experience with Spark optimization (partitioning, caching, cluster tuning).
- Knowledge of structured streaming in Databricks.
- Familiarity with Azure DevOps CI/CD pipelines for data platforms.
- Experience with Power BI.
- Experience migrating traditional ETL processes to cloud architectures.
Culture & Benefits
- Work from anywhere culture providing high flexibility.
- Comprehensive compensation package including health insurance.
- Explicit relocation program for eligible candidates.
- Professional development via certification programs and mentorship.
- Inclusive multicultural environment with regular team-building events.