Software Engineer, Data Infrastructure (Spark/Iceberg)

TL;DR

Software Engineer, Data Infrastructure (Data Infrastructure): Architecting and building systems to ingest, store, and serve massive volumes of real-time operational data with an accent on scalability and reliability at petabyte scale. Focus on designing high-throughput data integration platforms and optimizing distributed processing pipelines using Apache Spark and Iceberg.

Location: Must be located in San Francisco and open to working in office

Salary: $160,000 - $220,000 Annually

Company

hirify.global is an AI-enabled platform that transforms siloed data into operational intelligence for public safety and government agencies.

What you will do

• Design and operate a high-throughput, real-time data integration platform across diverse customer environments.
• Architect a scalable open table format layer for reliable data storage at petabyte scale.
• Build and optimize distributed data processing pipelines with Apache Spark and streaming technologies.
• Drive performance, reliability, and cost efficiency across the full data infrastructure stack.
• Collaborate with platform and product teams to define data contracts, schemas, and integration patterns.
• Establish organization-wide best practices and tooling for data infrastructure.

Requirements

• 2-5 years of experience operating large-scale data infrastructure systems in production.
• Expertise with open table formats, specifically Apache Iceberg (schema evolution, partitioning, compaction).
• Extensive hands-on experience with Apache Spark for batch and streaming data processing at scale.
• Background in real-time data integration using Apache Kafka, Apache Flink, or equivalents.
• Strong software engineering fundamentals in Python and/or Scala.
• Experience with AWS (S3-based data lakes), Kubernetes, and Airflow.
• Located in San Francisco and available for onsite work.

Culture & Benefits

• Competitive annual salary with equity and bonus opportunities.
• Comprehensive benefits package.
• Culture of ownership and collaboration, emphasizing empathy to improve user solutions.
• Opportunity to tackle hard technical problems and shape foundational infrastructure decisions.