Software Engineer - Data Pipelines (Cybersecurity)

TL;DR

Software Engineer - Data Pipelines (Python/AWS): Building and maintaining large-scale distributed data pipelines for exposure management and threat intelligence with an accent on data health, observability, and distributed tracing. Focus on optimizing PySpark jobs, designing Snowflake schemas, and automating production monitoring across 15+ regions.

Location: Onsite in Tel Aviv, Israel

Company

hirify.global is an Exposure Management company helping over 40,000 organizations worldwide understand and reduce cyber risk.

What you will do

• Own the full lifecycle of distributed data pipelines across 15+ regions.
• Develop large-scale distributed processing jobs to handle massive datasets.
• Build pipelines feeding the product's access layer, including threat intelligence, assets, and detections.
• Create frameworks for data health checks, conditional checks, and observability.
• Implement distributed tracing across all data pipelines.

Requirements

• 4+ years of experience building production data pipelines at scale.
• Proficiency in Python 3.10+ (Pydantic, async, decorators).
• Experience with Apache Airflow (DAG authoring, MWAA or self-hosted production).
• Expertise in PySpark (job authoring, optimization, EMR or Databricks).
• Strong knowledge of Snowflake (Snowpark, schema design) and AWS (S3, SQS/SNS, EMR).
• Strict testing discipline using pytest, moto, and integration tests.

Nice to have

• Experience with data quality tooling such as Great Expectations or Monte Carlo.
• Knowledge of CI/CD with artifact packaging and staged deployments.

Culture & Benefits

• End-to-end ownership of work without bureaucratic bottlenecks.
• AI-native development environment using custom plugins and protocols.
• High engineering rigor including dead-code detection and constant security scans.
• Automated on-call tooling designed to reduce operational pain.