4 дня назад
Data Engineer (Spark)
hhВакансия с HeadHunter. Контакт ведёт на hh.ru
Мэтч & Сопровод
Покажет вашу совместимость и напишет письмо
Описание вакансии
Текст:
TL;DR
Data Engineer (Spark): Building and optimizing end-to-end data pipelines, medallion-architecture models, and analytics-ready data marts with an accent on Spark performance, data quality, and scalable delivery. Focus on diagnosing multi-system pipeline failures, tuning partitions and shuffle behavior, and designing reliable ETL/ELT workflows across AWS, Databricks, Kafka, and dbt.
Location: Remote from Lithuania
Company
Nitka Technologies develops software for customers in the US and Europe and brings together approximately 300 professionals across multiple regions.
What you will do
- Lead discovery and requirements-gathering sessions with business stakeholders.
- Build, optimize, and maintain scalable ETL/ELT pipelines using dbt macros, incremental models, and snapshots.
- Design medallion architecture layers, foundational tables, star schemas, and reporting marts with SLAs and data quality contracts.
- Tune Spark jobs by optimizing partitions, reducing shuffle, and improving transformation patterns.
- Implement automated data quality checks with tools such as Great Expectations.
- Diagnose pipeline issues across Spark, Kafka, and dbt, and maintain technical documentation and operational runbooks.
Requirements
- 5+ years of data engineering experience.
- Advanced proficiency in Python, Spark, and SQL.
- Strong experience with AWS and Databricks.
- Familiarity with Kafka and modern data stack tools such as dbt and Airflow or equivalent.
- Deep understanding of dimensional modeling, medallion architecture, data warehouse design, and lambda, kappa, and lakehouse topologies.
- Spoken English at Intermediate level or higher required. AI fluency is also required for structuring problems and designing AI-assisted development pipelines.
Culture & Benefits
- 100% remote work from Lithuania.
- Full-time schedule of 8 hours per day.
- Long-term project with a California-based customer in the event ticketing industry.
- USD compensation.
- Paid vacation and holidays.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →