Senior Data Engineer (Healthcare AI)

TL;DR

Senior Data Engineer (Python/Spark): Building and scaling production-grade data pipelines to transform raw healthcare data into trusted datasets with an accent on canonicalization, pipeline reliability, and scalability. Focus on optimizing Spark jobs, implementing observability, and integrating complex external data sources to power ML models and product experiences.

Location: Must be based in the US

Salary: $180k–$220k

Company

hirify.global is a healthcare intelligence company providing an AI-powered platform to maximize financial outcomes and reduce costs for health plans.

What you will do

• Design and implement production-grade pipelines using Python, Spark SQL, and Airflow.
• Lead efforts to canonicalize raw healthcare data (claims, EHR) into internal models.
• Own the full lifecycle of core pipelines from ingestion to validated, queryable datasets.
• Onboard new customers by integrating raw data into internal pipelines and canonical models.
• Build resilient transformation logic with data quality checks, validation layers, and observability.
• Tune Spark jobs and optimize distributed processing performance.

Requirements

• 6+ years of experience as a Data Engineer building production-grade pipelines.
• Strong expertise in Python, Spark SQL, and Airflow.
• Experience processing large-scale file-based datasets (CSV, Parquet, JSON) in production.
• Proficiency with AWS or similar cloud platforms, including file storage and distributed compute.
• Ability to manage complex data workflows and communicate technical concepts to non-engineering partners.
• Must be based in the US

Nice to have

• Familiarity with healthcare data (837, 835, EHR, UB04, claims normalization).
• Experience building streaming pipelines using Kafka or SQS.

Culture & Benefits

• Digital-first environment with the ability to work from anywhere in the US.
• Full Medical, Dental, and Vision insurance for employees and their families.
• Unlimited Flexible Time Off (FTO).
• Competitive salary, equity, and 401(k) including employer match.
• Flexible and trusting work environment.