Senior Data Engineer (Batch & Streaming) – Analytics Platform

#vacancy Senior Data Engineer (Batch & Streaming) – Analytics Platform

Location: Remote EU - Ukrainian citizens only
Location EU : Poland | Also considered: Serbia, Montenegro
Rate: $5000-5800/month gross
Start: ASAP - Long-term engagement

About the Role:
We're building a greenfield analytics platform supporting both batch and real-time data processing. Looking for a Senior Data Engineer to design, implement, and evolve scalable data systems in AWS, combining hands-on development, architecture, and platform ownership.

Tech Stack:
• Apache Spark (Structured Streaming), PySpark
• AWS: S3, EMR, Glue, Athena, CloudWatch
• Streaming: Apache Kafka / Amazon Kinesis
• Lakehouse: Delta Lake
• IaC: Terraform / AWS CDK
• CI/CD: Docker, GitLab CI / GitHub Actions
• Observability: SparkUI, CloudWatch, monitoring best practices

Must Have:
• 5+ years in Data Engineering
• Strong hands-on experience with Apache Spark (batch + streaming)
• Proven experience building production data pipelines (batch & real-time)
• AWS data lake architecture experience (S3, EMR, Glue, Athena)
• Event-driven systems: Kafka or Kinesis
• Delta Lake or similar lakehouse formats
• Spark performance tuning (shuffle, skew, memory, partitioning)
• Cost optimization experience in AWS
• Docker + CI/CD + Infrastructure as Code (Terraform/CDK)

Nice to Have:
• Financial domain experience
• Spark on Kubernetes
• Data quality frameworks / metadata & lineage systems

Responsibilities:
🔸 Design & implement batch/streaming pipelines with Apache Spark
🔸 Build & evolve scalable AWS-based data lake architecture
🔸 Develop real-time event-driven processing systems
🔸 Own Spark workload performance tuning & cost optimization
🔸 Define best practices for data modeling, partitioning, schema evolution
🔸 Implement monitoring, observability & data quality controls
🔸 Contribute to infrastructure automation & CI/CD for data workflows
🔸 Mentor engineers & participate in architectural decisions


How to Apply: Fill out the candidate form - Показать контакты