Data Engineer (Spark)

TL;DR

Data Engineer (Spark): Develop and optimize complex Spark-based data pipelines and lakehouse architectures within the hirify.global platform with an accent on performance tuning, distributed data processing, and integration of SQL and Python workflows. Focus on designing scalable Spark environments, debugging PySpark code, and advising clients on architecture and best practices.

Location: Hybrid in Paris, Berlin, Amsterdam, Madrid, London

Company

hirify.global is The Universal AI Platform™ enabling organizations to control AI talent, processes, and technologies to build analytics, models, and agents with no-, low-, and full-code capabilities.

What you will do

• Design, build, and optimize data flows and Spark pipelines in hirify.global to improve performance and maintainability.
• Debug and enhance complex Spark code and data pipelines for reliability and efficiency.
• Guide clients on tuning and scaling Spark environments including Kubernetes and Databricks.
• Optimize SQL-based data pipelines and integrate diverse data workflows.
• Provide architectural guidance and best practices for Spark and lakehouse solutions.
• Collaborate with internal teams to resolve technical issues and contribute to knowledge sharing.

Requirements

• Location: Hybrid work in Paris, Berlin, Amsterdam, Madrid, or London
• Proven experience with Spark and PySpark in production environments.
• Strong skills in SQL, Python, and data pipeline optimization.
• Familiarity with lakehouse architectures and orchestration tools like Apache Airflow.
• Ability to communicate technical concepts effectively to diverse stakeholders.
• English: B2 level or higher required

Hiring process

• Initial call with Technical Recruiting team.
• Video interview with Field Engineer Hiring Manager.
• Technical assessment (home test) and debrief with team members.
• Final interview with VP Field Engineering.