Data Engineer (Databricks)

Location: Europe, Poland, Romania

Company

Software engineering company delivering a data engineering project for a deep-tech customer developing fusion-energy research platforms.

What you will do

• Design, develop, and maintain scalable data platforms and pipelines for analytics and operational use cases.
• Build distributed processing workflows with Apache Spark, PySpark, and Databricks.
• Develop cloud-native solutions using AWS and Azure, including data modeling and heterogeneous-source integration.
• Implement data quality controls, schema validation, lineage tracking, monitoring, and traceability.
• Optimize pipeline and infrastructure performance, scalability, reliability, and cloud costs.
• Collaborate with scientists, analysts, and engineers and contribute to CI/CD, infrastructure automation, and technology research.

Requirements

• 3+ years of commercial Data Engineering experience, including 2+ years of hands-on production experience with Databricks.
• Expertise with Databricks, Apache Spark, Delta Lake, medallion architecture, table optimization, Unity Catalog, Lakeflow Jobs/Workflows, and job optimization.
• Advanced Python, PySpark, and SQL skills.
• Strong AWS experience with S3, IAM, KMS, and at least one of AWS Batch, ECR, EventBridge, Lambda, or API Gateway.
• Experience designing event-driven and asynchronous pipelines, data quality frameworks, schema contracts, lineage, provenance, and platform monitoring.
• Strong knowledge of Infrastructure as Code and CI/CD, with practical Terraform and Databricks Asset Bundles experience.
• Experience with Git workflows, pull-request reviews, automated testing, OIDC or service-principal authentication, and secure access patterns.
• Practical MLflow experience covering experiment tracking, model registry, and reproducibility.
• Upper-Intermediate or higher English level.

Nice to have

• Docker packaging and batch workloads in AWS; familiarity with ECR and containerized data processing.
• Experiment optimization libraries such as scikit-optimize, BoTorch, Ax, or scipy.
• ML feature engineering and leakage prevention experience.
• Experience in regulated or high-security environments, including data governance, compliance, and data residency.
• Experience disabling or restricting AI-assisted tooling in secure environments.

Culture & Benefits

• Work in a research-intensive fusion-energy and deep-tech environment.
• Collaborate with scientists and engineers on innovative scientific initiatives.
• Engineering culture focused on technical excellence, long-term partnerships, and continuous learning.
• Use modern distributed data technologies and work with skilled global teams.