TL;DR

Distributed Systems Software Engineer (Big Data): Developing reliable and efficient data infrastructure using Spark, Trino, Airflow, and Kafka to support scalable data processing and analytics with an accent on real-time and batch data workflows. Focus on designing resilient distributed systems that manage thousands of compute nodes across multiple data centers, ensuring scalability and high availability.

Location: Must be based in the USA

Company

%hirify_global% is a technology-driven company focused on enabling scalable, high-performance data processing and analytics.

What you will do

Build scalable data processing and analytics services utilizing big data stack.
Design, develop, and operate resilient distributed systems that manage thousands of compute nodes across multiple data centers.
Resolve complex technical challenges and drive innovations that enhance system resilience, availability, and performance.
Manage the full lifecycle of services, balancing live-site reliability, feature development, and technical debt retirement.
Participate in the team’s on-call rotation to address complex, real-time issues, keeping critical services operational and highly available.
Provide mentorship and technical guidance to junior engineers, fostering growth, collaboration, and knowledge-sharing within the team.

Requirements

Bachelor’s or Master’s in Computer Science, Engineering, or a related field with 5+ years of experience in distributed systems, big data, or similar roles.
Proficiency in cloud platforms (AWS, GCP, or Azure), containerization (Docker, Kubernetes), and infrastructure-as-code (Terraform, Ansible).
Hands-on experience with Hadoop, Spark, Trino (or similar SQL query engines), Airflow, Kafka, and related ecosystems.
Strong skills in Python, Java, Scala, or other programming languages relevant to distributed systems.
Solid knowledge of distributed computing principles, data partitioning, fault tolerance, and performance tuning.
Proven ability to troubleshoot complex system issues, optimizing for speed, efficiency, and scale.