Software Engineer, Spark Platform

Location: Hybrid; must be located in or willing to relocate to San Francisco, Sunnyvale, Seattle, or New York City, United States

Salary: $130,600–$192,000 USD base salary per year, plus potential equity grants

Company

hirify.global operates a large-scale technology platform and provides data, analytics, and machine learning infrastructure across the company.

What you will do

• Build and operate the in-house Apache Spark platform, including the runtime, scheduler, reliability systems, and user-facing tooling.
• Develop multi-tenant scheduling, executor bin-packing, and cost-aware placement for workloads serving dozens of consumer teams.
• Own cluster lifecycle automation for provisioning, upgrades, capacity changes, and node-failure handling.
• Build observability and incident automation to improve end-to-end debugging and on-call sustainability.
• Collaborate with senior engineers and platform consumers on shuffle, runtime, and systems architecture.

Requirements

• B.S., M.S., PhD in Computer Science, or equivalent experience.
• 2+ years of industry experience operating production distributed systems.
• Experience operating Apache Spark at scale on Amazon EMR, Databricks, or an in-house deployment.
• Hands-on experience with production Kubernetes systems, including controllers, operators, custom resources, and multi-tenant cluster failure modes.
• Familiarity with batch or big-data schedulers, Spark-on-Kubernetes operators, observability stacks, SLOs, and SLIs.
• Professional experience with Python, Go, Scala, or Java; SQL fluency; and comfort with AWS cloud infrastructure.

Culture & Benefits

• Hybrid work environment with platform collaboration across the company.
• 401(k) plan with employer matching.
• Medical, dental, vision, disability, and basic life insurance.
• 16 weeks of paid parental leave, paid time off, paid sick leave, and 11 paid holidays.
• Wellness, commuter, family-forming assistance, and mental health benefits.
• Flexible paid time off for salaried employees.