Senior Data Platform Engineer II (AI)

TL;DR

Senior Data Platform Engineer II (AI): Define and evolve an AI-first real-time and batch data platform built on AWS and Databricks with an accent on streaming systems, lakehouse design, and data-serving layers optimized for AI and ML. Focus on architecting scalable, reliable foundations for real-time intelligence, feature generation, and decisioning systems.

Location: New York, NY

Company

Fastest growing healthcare technology company building products to make prescriptions accessible and affordable via pharma-to-patient cloud platforms like BlinkRx.

What you will do

• Own end-to-end AI-first data platform architecture spanning streaming ingestion, lakehouse storage, feature generation, and serving layers
• Architect real-time streaming systems using AWS Kinesis and Spark Structured Streaming for low-latency AI/ML inference and decisioning
• Design stream-to-lakehouse patterns unifying real-time and historical data for analytics and ML from a single source of truth
• Build data, metrics, and feature APIs exposing AI-ready datasets for applications, analytics, and machine learning
• Establish canonical event schemas, feature definitions, and data contracts for event-driven systems and model training
• Drive platform modernization, optimize AWS and Databricks infrastructure for AI scalability, cost efficiency, and high-performance workloads

Requirements

• Deep expertise in real-time and distributed data systems including AWS Kinesis and Spark Structured Streaming
• Strong command of Databricks on AWS (Delta Lake, clusters, jobs) and core AWS services (S3, IAM, networking)
• Proven experience designing data- and feature-serving architectures and APIs for analytics, metrics, and ML consumption
• Advanced Python and SQL skills for scalable, high-performance data pipelines
• Ability to design idempotent, replayable, observable, and AI-ready data platforms at scale

Culture & Benefits

• Highly collaborative team of builders and operators inventing new ways in healthcare innovation
• Impact millions of patients at intersection of healthcare and finances
• Relentlessly learning, curious, and aggressively collaborative cross-functional environment
• Equal opportunity employer valuing diversity