6 дней назад
Middle Data Engineer (Databricks)
Вакансия из Hirify Global, списка международных tech-компанийДля мэтча и отклика нужен Plus
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Middle Data Engineer (Databricks): Building and maintaining scalable ETL/ELT data pipelines and architectures for structured, unstructured, batch, and real-time data with an accent on data quality, governance, and multi-source processing. Focus on optimizing large-scale pipelines, implementing secure data lineage and access controls, and solving reliability challenges across heterogeneous cloud data environments.
Location: Remote
Company
develops IT solutions for clients in the US and European markets and has more than 170 IT professionals.
What you will do
- Design, develop, and maintain scalable ETL/ELT pipelines for structured and unstructured data.
- Create data architectures supporting frequent transformations and complex, multi-source environments.
- Collaborate with data scientists, analysts, and stakeholders to align pipelines with business requirements.
- Monitor, troubleshoot, and optimize batch and real-time data processing systems.
- Implement data governance, security, privacy, lineage, access control, and auditability practices.
- Evaluate emerging data engineering technologies, AI frameworks, and infrastructure improvements.
Requirements
- At least 2 years of data engineering experience.
- Experience building automated, scalable data pipelines using ETL/ELT methodologies, including Databricks.
- Strong Python, PySpark, SparkSQL, and SQL skills.
- Experience with relational and NoSQL databases, APIs, streaming data such as Apache Kafka, and unstructured formats.
- Hands-on experience with Apache Spark and at least one cloud platform: IBM Cloud, AWS, Azure, or Google Cloud.
- Knowledge of Apache Airflow or Dagster, data quality, GDPR/CCPA compliance, encryption, and role-based access control.
Culture & Benefits
- Remote full-time work.
- Collaboration with data scientists, analysts, and other stakeholders.
- Work on technology-enabled solutions for higher education and nonprofit organizations.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →