Spark Scala Developer (Apache Spark)

TL;DR

Spark Scala Developer (Apache Spark): Designing and optimizing big data applications and scalable data pipelines with an accent on distributed computing, performance optimization, and data quality. Focus on architecting real-time and batch processing solutions, troubleshooting performance issues, and mentoring junior developers.

Location: Hybrid in San Jose, California, United States

Salary: $220,800–$331,200 per year

Company

hirify.global is a global technology and consulting corporation focused on AI-powered, cloud-native software solutions and an AI and data platform.

What you will do

• Design, develop, and optimize big data applications using Apache Spark and Scala.
• Architect and implement scalable data pipelines for both batch and real-time processing.
• Optimize Spark jobs for performance and cost-effectiveness on distributed clusters.
• Implement data quality checks, logging, monitoring, and alerting for ETL jobs.
• Mentor junior developers and lead code reviews.
• Troubleshoot and resolve performance issues and bugs in big data solutions.

Requirements

• 12+ years of total software development experience.
• Minimum 5+ years of hands-on experience with Apache Spark and Scala.
• Strong experience with distributed computing, parallel data processing, and cluster computing frameworks.
• Proficiency in Scala with deep knowledge of functional programming.
• Solid understanding of Spark tuning, partitions, joins, broadcast variables, and performance optimization techniques.
• Experience with cloud platforms such as AWS, Azure, or GCP (especially EMR, Databricks, or HDInsight).
• Hands-on experience with Kafka, Hive, HBase, NoSQL databases, and data lake architectures.
• Familiarity with CI/CD pipelines, Git, Jenkins, and automated testing.

Nice to have

• Experience with Databricks, Delta Lake, or Apache Iceberg.
• Exposure to machine learning pipelines using Spark MLlib or integration with ML frameworks.
• Experience with data governance tools (e.g., Apache Atlas, Collibra).
• Contributions to open-source big data projects.
• Excellent communication and and leadership skills.

Culture & Benefits

• Competitive and comprehensive benefits program including healthcare, 401(k), pension plan, and employee stock purchase plan.
• Generous paid time off including 12 holidays, 56 hours sick time, 120 hours vacation, and 12 weeks parental bonding leave.
• Access to an AI-driven learning platform for skill development and industry-recognized certifications.
• Diverse and inclusive employee resource groups, volunteering opportunities, and retail discounts.
• Work in an environment that fosters growth, curiosity, collaboration, trust, and support.
• Focus on client success, innovation, trust, and personal responsibility as core values.