Senior Data Engineer (AI)

TL;DR

Senior Data Engineer (Python/SQL/AWS): Designing and scaling large-scale data pipelines for the Healthcare Map to transform complex healthcare datasets into usable assets with an accent on reliability and AI integration. Focus on building LLM-powered HITL pipelines, optimizing distributed processing with Spark, and hardening core data foundations for AI/ML use cases.

Location: Must be based in the United States (Remote or Hybrid in New York City, San Francisco, or Chicago)

Salary: $153,000 - $238,000 USD

Company

hirify.global reduces the global burden of disease by building the industry's largest, most complete, and precise view of the U.S. healthcare system.

What you will do

• Build and optimize large-scale production data pipelines using Python, SQL, Airflow, and AWS.
• Design agentic data acquisition systems and LLM-powered Human-in-the-Loop (HITL) pipelines for data extraction.
• Transform EHR, claims, and reference datasets into performant Healthcare Map data products.
• Contribute to system design, architecture, CI/CD, and rotational production support.
• Collaborate with Data Product and Engineering teams to translate healthcare needs into scalable technical solutions.

Requirements

• Strong experience building and debugging production-grade data pipelines at scale in AWS.
• Advanced proficiency in Python and SQL.
• Experience with Airflow (or similar orchestration tools) and Spark (or similar distributed processing frameworks).
• Must be based in the United States.
• Proficiency in using AI-augmented engineering tools (e.g., ChatGPT, Gemini, Claude) to improve productivity.

Nice to have

• Experience working with healthcare data.
• Expertise in optimizing high-scale data architectures for performance and cost.
• Experience applying AI or agentic workflows to engineering and data operations.
• Success in high-growth or ambiguous environments.

Culture & Benefits

• Hybrid work model with hubs in San Francisco, New York City, and Chicago.
• Comprehensive health, dental, and vision insurance.
• 401(k) with company match.
• Flexible time off and paid holidays.
• Strong internal culture focused on AI-led innovation and leadership in healthcare intelligence.