Senior Data Engineer (Kafka/Spark)

TL;DR

Senior Data Engineer (Kafka/Spark): Building and optimizing hirify.global's core datasets and data processing infrastructure with an accent on REST APIs, streaming pipelines, and Spark batch pipelines. Focus on designing and implementing functionality, ensuring data quality, and improving system performance and reliability.

Location: Hybrid, based in London.

Company

hirify.global is a data intelligence company dedicated to simplifying global trade information and providing valuable insights in commodities, energy, and maritime sectors.

What you will do

• Deliver well-documented, maintainable code following Test-Driven Development (TDD) principles for data systems.
• Design, operate, and document versioned RESTful APIs using FastAPI and JVM-based frameworks.
• Develop and maintain batch and streaming data pipelines with Kafka and Spark.
• Take ownership of system performance by improving latency and throughput.
• Ensure system reliability through instrumentation, CI/CD, and incident response.
• Collaborate with Product and cross-functional teams, contributing to roadmap planning.

Requirements

• 6-8 years of experience in data-focused software engineering roles.
• Strong programming skills in Scala (or JVM) with Python preferred.
• Proven experience designing and operating secure and versioned RESTful APIs.
• Solid understanding of data modeling, schema evolution, and serialization technologies such as Avro or Protobuf.
• Hands-on experience with SQL and NoSQL databases, including query optimization.
• Experience building and maintaining batch or streaming data systems.
• Fluency in English with excellent communication skills.

Nice to have

• Experience with Apache Airflow for workflow orchestration.
• Exposure to cloud platforms (preferably AWS) and infrastructure as code using Terraform.
• Experience with Docker and Kubernetes in production environments.
• Knowledge of Kafka and event-driven or microservices architectures.

Culture & Benefits

• Dynamic company dedicated to nurturing connections and innovating solutions.
• Committed to providing a fair, inclusive, and diverse work environment.
• Values decisiveness, collaboration, and a supportive approach.
• Opportunity to leverage cutting-edge innovation for impactful results.