Technical Lead, Spark (Java)

TL;DR

Technical Lead, Spark (Java): Building enterprise-grade distributed data processing systems for hirify.global’s Apache Spark and Livy distribution with an accent on scaling Spark workloads across thousands of nodes and delivering features from prototypes to production at scale. Focus on distributed systems design, debugging deployment and failures, and deep work on Spark/SQL planning plus data layout and table formats like Parquet and Iceberg.

Location: Spain (Barcelona) — Remote

Company

hirify.global builds enterprise data platform technology that helps customers store, manage, and process data at massive scale.

What you will do

• Design and deliver new features for the data engineering experience, taking them from prototypes to production at scale.
• Contribute to Apache Spark and Livy.
• Develop features using Scala/Java/Python on modern platforms.
• Work across distributed data processing components, including SQL planners/optimizers, data layout, Parquet and Iceberg, and fault tolerance.
• Debug system-level deployment issues, perform root-cause analysis, and resolve production failures.
• Improve internal infrastructure and collaborate with distributed stakeholders.

Requirements

• 6+ years of professional software development.
• Experience leading and delivering complex product enhancements.
• Strong understanding of at least one of: Java, Scala, or Python.
• Experience with systems design and development.
• Strong distributed systems experience.
• Strong oral and written communication skills; ability to research and solve problems independently.

Nice to have

• Experience with SQL planners.
• Experience using/developing Apache Spark, Livy, or related technologies.
• Experience designing and developing large-scale distributed systems with performance and scheduling considerations.

Culture & Benefits

• Generous PTO policy and support for work-life balance with Unplugged Days.
• Flexible WFH policy.
• Mental & physical wellness programs.
• Phone and internet reimbursement program.
• Access to continued career development and comprehensive benefits.
• Employee Resource Groups and paid volunteer time.

Hiring process

• Interviews focused on distributed systems and Spark/Livy experience.
• Technical evaluation of system design, debugging, and feature delivery approach.
• Collaboration and communication assessment with cross-functional stakeholders.