Data Engineer (Databricks)
Вакансия из Hirify Global, списка международных tech-компанийДля мэтча и отклика нужен Plus
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
TL;DR
Data Engineer (Databricks): Designing and building scalable data infrastructure for a Master Data Management platform with an accent on Databricks Lakehouse technologies and real-time streaming. Focus on developing high-performance batch and streaming pipelines, optimizing Spark workloads, and ensuring data quality at scale.
Location: Must be based in Medellin, Colombia or Argentina
Company
is a global staffing company providing specialized engineering talent for innovative SaaS platforms.
What you will do
- Lead the design and development of robust data pipelines within the Databricks Lakehouse Platform.
- Implement real-time data ingestion and processing using Delta Live Tables and Change Data Capture patterns.
- Optimize PySpark workloads to ensure efficient processing of large-scale datasets.
- Design logical and physical data models to support analytical and operational requirements.
- Ensure data reliability and consistency through idempotent pipeline design and automated testing.
- Collaborate with AI/ML engineers and data scientists to integrate data sources and support intelligent features.
Requirements
- 5+ years of experience as a Data Engineer in production environments.
- Deep expertise with Databricks Lakehouse Platform (Delta Lake, Unity Catalog, Databricks SQL).
- Proven proficiency in Apache Spark and PySpark for complex transformations.
- Demonstrated experience with streaming technologies and Delta Live Tables.
- Strong understanding of CDC/CDF patterns and dimensional data modeling (Kimball).
- Excellent communication skills for technical and non-technical stakeholder collaboration.
Nice to have
- Experience with Entity Resolution or MDM systems.
- Knowledge of cloud platforms like AWS or Azure.
- Familiarity with MLOps practices and CI/CD for data pipelines.
Culture & Benefits
- Opportunity to work on a cutting-edge AI-ready data platform.
- Global staffing environment with professional growth opportunities.
- Focus on modern data engineering best practices and high-performance systems.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →