Staff Product Support Engineer (Hadoop)

TL;DR

Staff Product Support Engineer (Hadoop): Designing, optimizing, and scaling Hadoop and Spark-based data processing systems with an accent on system reliability and performance tuning for high-throughput applications. Focus on troubleshooting complex data pipelines, managing cluster operations, and ensuring stability during migrations to new ODP environments.

Location: Must be based in California, United States (On-site)

Company

hirify.global provides enterprise data observability solutions to help organizations monitor, troubleshoot, and optimize their complex data ecosystems.

What you will do

• Design and optimize distributed Hadoop-based applications for low-latency and high-throughput big data workloads.
• Provide expert-level troubleshooting for data or performance issues within Hadoop, Nifi, and Spark clusters.
• Perform deep-dive performance analysis and debugging of Nifi, Impala, and Spark jobs to reduce resource consumption.
• Collaborate with infrastructure teams to manage large-scale clusters on Kubernetes and cloud platforms like AWS EMR or GCP Dataproc.
• Ensure the stability of ODP Hadoop environments throughout migration processes and post-migration optimizations.
• Participate in rotational 24/7 support coverage to address customer demands and system reliability.

Requirements

• Must be able to work in a 24/7 rotational shift environment as required.
• Deep expertise in managing Hadoop, Nifi, and Spark core components.
• Proven experience with performance tuning and debugging of large-scale distributed data pipelines.
• Proficiency in Linux, command-line operations, and scripting (Bash, Python, or Scala).
• Experience managing multi-node clusters and virtual machine environments.
• Solid understanding of application, server, and network security management.

Nice to have

• Master’s degree in a technical field.
• Experience with cloud provider certifications (AWS, Azure, GCP) or Kubernetes.
• Working knowledge of SQL databases such as MySQL or PostgreSQL.
• Experience performing Proof of Concepts (POC) and assisting with software deployments.