Principal Data Platform Engineer (Healthcare)

TL;DR

Principal Data Platform Engineer (AWS/Databricks): Define and evolve real-time and batch data platform built on AWS and Databricks with an accent on streaming ingestion, lakehouse storage, and data-serving layers. Focus on architecting low-latency streaming systems for analytics and machine learning use cases, unifying real-time and historical data, and establishing scalable APIs and event schemas.

New York, NY

Company

Fastest growing healthcare technology company building products like BlinkRx and Quick Save to make prescriptions accessible and affordable.

What you will do

• Define and own end-to-end data platform architecture spanning streaming, lakehouse, and serving layers
• Architect real-time systems using AWS Kinesis and Spark Structured Streaming for low-latency workloads including ML
• Design stream-to-lakehouse patterns unifying real-time and historical data with correctness guarantees
• Build data and metrics APIs for downstream applications, analytics, and machine learning systems
• Establish event schemas and data contracts for event-driven and API consumption
• Drive modernization, optimize AWS and Databricks infrastructure, and lead platform initiatives

Requirements

• 10+ years expertise in real-time and distributed data systems including AWS Kinesis and Spark Structured Streaming
• Strong command of Databricks on AWS (Delta Lake, clusters, jobs) and AWS services (S3, IAM, networking)
• Experience designing data-serving architectures and APIs for analytics and ML workflows
• Advanced Python and SQL for scalable data pipelines
• Ability to design idempotent, replayable, observable data platforms at scale

Culture & Benefits

• Highly collaborative team of builders and operators inventing new ways in healthcare
• Impact millions of patients at intersection of healthcare and finances
• Relentlessly learning, curious, and aggressively collaborative cross-functional environment
• Equal opportunity employer valuing diversity