Data Engineer (AWS, Databricks)

TL;DR

Data Engineer (AWS, Databricks): Building and optimizing high-load data pipelines and services for a cloud-native Lakehouse platform with an accent on migration from SQL Server, data quality, and scalability. Focus on designing resilient architectures, collaborating for analytics and ML use cases, and implementing best practices for data governance and security.

Location: This role offers remote work from **Bulgaria, Georgia, Kazakhstan, or Poland**. Onsite work is also available in offices located in Almaty, Astana, Cluj-Napoca, Krakow, Larnaca, Lodz, Lublin, Riga, Sofia, Tbilisi, Varna, Warsaw, Wroclaw, and Yerevan.

Company

hirify.global is a global technology consulting company looking for a Data Engineer to work with a client, one of the largest betting communities and pioneers of the betting exchange model.

What you will do

• Migrate legacy SQL Server workloads to a modern Lakehouse architecture on AWS and Databricks.
• Design, build, and maintain data pipelines for batch and real-time processing.
• Ensure data quality, reliability, and scalability across all pipelines and services.
• Collaborate with data scientists, analysts, and business stakeholders to deliver data solutions for analytics and ML use cases.
• Implement best practices for data governance, security, and compliance.
• Optimize performance and cost efficiency in a cloud-native environment.

Requirements

• Strong proficiency in Python for data engineering tasks.
• Hands-on experience with AWS services (e.g., S3, Glue, Lambda, EMR).
• Expertise in Databricks and Spark for big data processing.
• Solid understanding of SQL and relational database concepts.
• Experience with ETL/ELT frameworks and workflow orchestration tools (e.g., Airflow).
• Knowledge of data modeling, data warehousing, and Lakehouse principles.

Nice to have

• Familiarity with streaming technologies (Kafka, Kinesis).
• Experience with CI/CD pipelines for data solutions.
• Understanding of data security and compliance in cloud environments.
• Exposure to machine learning workflows and MLOps concepts.

Culture & Benefits

• Vacation as per the laws of your country.
• Assistance with health insurance for you and your loved ones.
• 10 days of sick pay without a doctor's note, subsequent days as per local laws.
• Time off for state holidays according to the official calendar.
• Pleasant environment with corporate parties and get-togethers.
• Comfort service for technical and everyday work problems.