Senior Big Data Engineer (Scala/Spark)

TL;DR

Senior Big Data Engineer (Scala/Spark): Responsible for developing data pipelines, maintaining infrastructure, and optimizing performance across distributed systems with an accent on accuracy, efficiency of statistical algorithms and infrastructure. Focus on collaboration with global engineering teams on distributed data architecture, data lakes, warehousing solutions, and CI/CD workflows.

Location: Fully remote from Portugal

Company

hirify.global is a leading provider of technology consulting, platform and product engineering, AI, and advanced analytics services.

What you will do

• Design, develop, and maintain scalable data processing pipelines using Apache Spark and Scala.
• Ensure accuracy and efficiency of statistical algorithms implemented in production.
• Collaborate with global engineering teams on distributed data architecture and design.
• Work with large-scale distributed systems (e.g., Hadoop, Kafka, Cassandra) for data ingestion and processing.
• Contribute to the build and optimization of Data Lakes and Data Warehousing solutions.
• Support and enhance CI/CD workflows, infrastructure automation, and deployment processes.

Requirements

• Proficiency in Scala/Spark.
• Hands-on expertise with Hadoop.
• Processing and computation frameworks: Kafka, Spark, Dataframe/SQL API.
• DB engines: Oracle, Postgres, Teradata, Cassandra.
• Implementing Data Lakes, Data Warehousing or analytics systems is an advantage.
• Good understanding of distributed computing technologies, approaches, and patterns.

Nice to have

• DevOps/CICD skills

Culture & Benefits

• Opportunity to work on bleeding-edge projects.
• Work with a highly motivated and dedicated team.
• Competitive salary
• Flexible schedule
• Professional development opportunities