Lead Data Engineer (AI)

Location: Remote/Hybrid within the United States; preferably Boston, MA. Occasional travel to hirify.global offices may be required.

Salary: $142,300–$195,700 per year, plus bonus eligibility.

Company

hirify.global is a large U.S. healthcare and insurance company providing healthcare services to Medicare, Medicaid, families, individuals, and communities.

What you will do

• Own end-to-end delivery for the data engineering pod, including planning, execution, quality, and operational readiness.
• Design and evolve the NBA Databricks Lakehouse using Bronze, Silver, and Gold layer standards, lineage, and governance patterns.
• Build scalable batch and real-time pipelines for member, clinical, claims, behavioral, engagement, and socioeconomic datasets.
• Develop feature tables and reusable data products supporting model training, scoring, reinforcement learning, and decision intelligence.
• Design Kafka and Spark Structured Streaming architectures for near real-time decision-making.
• Lead engineers and contractors while partnering with data science, AI engineering, decision intelligence, and platform engineering teams.

Requirements

• Bachelor's degree in computer science or a related field.
• 7+ years of data engineering experience, including 1–2 years in a lead or technical leadership role; alternatively, 8+ years of progressive IT experience and 2+ years of project leadership experience.
• Expert SQL and Python skills with significant experience operating Spark-based data platforms.
• Deep experience with Databricks, Delta Lake, Lakehouse architectures, and enterprise-scale Bronze/Silver/Gold implementations.
• Experience with batch and streaming pipelines, data quality engineering, validation, lineage, monitoring, and operational support.
• Experience designing secure and compliant data platforms in regulated environments, including PHI and HIPAA requirements.

Nice to have

• Experience with Databricks Feature Store, Delta Live Tables, Unity Catalog, and Databricks Workflows.
• Kafka, event streaming, real-time feature engineering, feature-serving architectures, or model-data integration experience.
• Experience with Azure Data Factory, Azure Event Hubs, Azure Data Lake Storage, or other Azure data services.
• Experience with data observability tools such as Great Expectations or Monte Carlo.
• Healthcare, insurance, machine learning, reinforcement learning, recommendation engine, or decision intelligence experience.

Culture & Benefits

• Remote or hybrid work with a dedicated workspace and internet service of at least 25 Mbps download and 10 Mbps upload.
• Typical schedule is Monday–Friday, 8 hours per day, 40 hours per week, with some flexibility based on business needs.
• Medical, dental, and vision coverage, 401(k), paid time off, holidays, parental and caregiver leave, disability coverage, and life insurance.
• Use of modern AI development tools such as Claude and GitHub Copilot is expected.
• Occasional travel may be required for training, meetings, or conferences.

Hiring process

• Selected applicants complete a 10–15 minute on-demand HireVue assessment.
• Applicants who advance are invited to subsequent interview rounds.
• A formal offer requires entering an SSN in Workday for duplicate-profile screening.