Data Engineer (Databricks)

Location: Remote, Poland; Warsaw, Masovian Voivodeship

Company

Builds engineering solutions and long-term partnerships, contributing to a cloud-based data platform for scientific research and operational workflows in the fusion energy domain.

What you will do

• Design, develop, and maintain scalable data platforms and pipelines for analytics and operational use cases.
• Build distributed processing workflows with Apache Spark, PySpark, and Databricks across AWS and Azure.
• Model and optimize data structures, integrating heterogeneous sources with strong quality, consistency, lineage, and traceability controls.
• Implement schema validation, monitoring, infrastructure automation, CI/CD, and reliability and cost optimization practices.
• Collaborate with scientists, analysts, and engineers to support experimentation, analytics, and machine-learning workflows.
• Research and introduce technologies and approaches within the data engineering ecosystem.

Requirements

• 3+ years of commercial Data Engineering experience, including 2+ years of hands-on production experience with Databricks.
• Strong expertise in Delta Lake, medallion architecture, table optimization, Databricks Unity Catalog, Lakeflow Jobs/Workflows, and job optimization.
• Advanced Python, PySpark, and SQL skills.
• Hands-on AWS experience with S3, IAM, KMS, and at least one of AWS Batch, ECR, EventBridge, Lambda, or API Gateway.
• Experience designing event-driven and asynchronous pipelines, with strong Infrastructure as Code and CI/CD knowledge.
• Practical experience with Terraform, Databricks Asset Bundles, Git workflows, pull-request reviews, and automated testing.
• Understanding of secure authentication approaches including OIDC and service principals.
• Practical MLflow experience covering experiment tracking, model registry, and reproducibility.
• Strong understanding of data quality, lineage, schema contracts, provenance, and data quality frameworks such as Great Expectations, Soda, pyDeequ, or Databricks DQX.
• Experience monitoring and optimizing data platform reliability and cloud costs; Upper-Intermediate or higher English required.

Nice to have

• Experience packaging workloads with Docker and running batch workloads in AWS; familiarity with ECR and containerized data processing.
• Experience with scikit-optimize, BoTorch, Ax, or scipy for experiment optimization.
• Understanding of ML feature engineering and leakage prevention.
• Experience in regulated or high-security environments, including data governance, compliance, and data residency.
• Experience disabling or restricting AI-assisted tooling or features in secure environments.

Culture & Benefits

• Full-time remote work associated with Poland.
• Work in a research-intensive fusion-energy R&D environment with scientists and engineers.
• Focus on engineering excellence, long-term partnerships, and continuous learning.
• Use modern distributed data technologies within global engineering teams.