Consultant Data Engineer (AI/ML)

TL;DR

Consultant Data Engineer (Spark/Python): Building and operating data-driven and intelligent systems in production environments with an accent on Spark optimization and AI/ML integration. Focus on designing scalable data pipelines and implementing GenAI concepts into production-grade software.

Location: Remote within Ukraine (Kyiv, Lviv, Rivne, Odesa)

Company

A global business and technology transformation partner helping organizations accelerate their transition to a digital and sustainable world.

What you will do

• Build and operate production-grade data-driven and intelligent systems.
• Develop and optimize large-scale workloads using Apache Spark and Spark SQL.
• Implement Spark Structured Streaming for real-time data processing.
• Integrate Machine Learning and GenAI components into scalable data platforms.
• Utilize Terraform and Databricks Asset Bundles for infrastructure management.
• Collaborate on defining priorities and driving technical execution in a high-performance environment.

Requirements

• Strong expertise in Apache Spark, including optimization, Structured Streaming, and Spark SQL.
• Proficiency in Python for data processing, ML, and AI use cases.
• Practical experience in ML model development and integration into production.
• Solid understanding of AI and Generative AI concepts.
• Experience with Terraform and Databricks Asset Bundles.
• Must be based in Ukraine.

Nice to have

• Deep expertise in Databricks and Databricks Apps.
• Comfort working in Linux environments.
• Background in financial markets, such as market making or trading.
• Knowledge of German or French.

Culture & Benefits

• Health insurance provided from the first day of employment.
• Company-paid Christmas holidays from December 25 to December 31.
• Strong commitment to supporting military and veterans via Veteran HUB and Superhumans center.
• Opportunity to work in a diverse global group of 340,000 team members across 50 countries.