5 дней назад
Senior Big Data Engineer (Scala, Kafka)
Вакансия из Hirify Global, списка международных tech-компанийДля мэтча и отклика нужен Plus
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Senior Big Data Engineer (Scala, Kafka): Building scalable streaming and batch data pipelines for a geospatial IoT telemetry platform with an accent on telemetry parsing, enrichment, and unified domain modeling. Focus on designing vendor protocol parsers, extending Apache Flink and Spark pipelines, and validating large-scale telemetry data with Delta Lake and Databricks.
Location: Ukraine; remote or office work available.
Company
is hiring for a geospatial IoT telemetry platform processing data from agricultural and construction machinery.
What you will do
- Lead architecture discovery and design for telemetry-enrichment mechanisms, including requirements gathering, data-flow mapping, and target architecture proposals.
- Design and implement parsers for vendor IoT protocols, converting raw device payloads into a unified telemetry model.
- Extend Apache Flink streaming pipelines with IoT features, signal-catalog updates, and production fixes.
- Work across Apache Flink streaming and Apache Spark, Delta Lake, HBase, and GeoMesa batch and persistence workflows.
- Develop unit and pipeline-level tests using ScalaTest and local Flink and Spark environments.
- Analyze telemetry data in Databricks, participate in code reviews, and collaborate with downstream API and application teams through Azure DevOps.
Requirements
- Strong production experience with Scala in software or data engineering.
- Experience building and maintaining large-scale pipelines with Apache Flink and/or Apache Spark.
- Experience with event-driven and streaming technologies such as Azure Event Hubs or Apache Kafka.
- Understanding of streaming and batch processing architectures.
- Experience with Delta Lake, Databricks, and distributed data stores such as HBase or GeoMesa.
- Strong software engineering practices, including automated testing, code reviews, and CI/CD; upper-intermediate or higher English required.
Nice to have
- Experience with Apache Pekko or similar backend and microservice technologies.
Culture & Benefits
- Remote or office work format.
- Collaboration with downstream API and application teams.
- PR-based delivery using Azure DevOps.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →