Назад
Company hidden
2 дня назад

Lead Data Engineer (P3999)

125 000 - 207 000$
Формат работы
onsite
Тип работы
fulltime
Грейд
lead
Английский
b2
Страна
US
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Lead Data Engineer (Azure/Databricks): Designing and building batch and event-driven data solutions, streaming pipelines, APIs, and data products on Azure with an accent on Databricks, Spark, Python, SQL, and reliable event processing. Focus on leading blended engineering teams, solving complex streaming and schema-evolution challenges, optimizing production pipelines, and applying governance, CI/CD, and AI-assisted development practices.

Location: Cincinnati, OH or Chicago, IL, United States; 5-day in-office work schedule required.

Salary: $125,000–$207,000 USD annually, plus eligibility for variable compensation.

Company

hirify.global is a retail data science, insights, and media company using first-party retail data from more than 62 million U.S. households to support personalized shopper experiences, loyalty marketing, and retail media advertising.

What you will do

  • Lead the design and development of Databricks-based data solutions and provide day-to-day technical direction to blended onshore/offshore engineering teams.
  • Design and build batch and event-driven data solutions, including streaming ingestion, event processing, publishing, event contracts, schemas, versioning, and schema evolution.
  • Develop, maintain, monitor, and optimize data pipelines handling ordering, late-arriving data, duplicates, retries, dead letters, replay, backfill, latency, and consumer lag.
  • Design data models, workflows, operational procedures, quality checks, automated tests, monitoring, error handling, governance, privacy, security, and Unity Catalog lineage.
  • Build CI/CD pipelines, infrastructure as code, environment promotion practices, and REST APIs hosted in Kubernetes.
  • Partner with Product, Data Science, and technical architects; support model and feature pipelines, respond to incidents, perform root cause analysis, mentor junior engineers, and improve engineering practices.

Requirements

  • Bachelor’s degree in Computer Science, Management Information Systems, Mathematics, Business Analytics, or another technically strong program, or equivalent experience.
  • 5+ years of professional data engineering experience, including 5+ years with Spark/Databricks/PySpark and 5+ years with Python and SQL.
  • 2+ years technically leading or mentoring engineers, including blended onshore/offshore or vendor teams.
  • Experience with event-driven or streaming data solutions using Spark Structured Streaming and Kafka, Azure Service Bus, or a similar messaging platform.
  • Strong understanding of publish/subscribe, event sourcing, change data capture, delivery guarantees, idempotent processing, windowing, watermarking, and out-of-order or late-arriving events.
  • Experience with event schema management using Avro, Protobuf, or JSON Schema; backward-compatible schema evolution; and data modeling including dimensional modeling, medallion architecture, Delta Lake, and schema evolution.
  • Experience with Azure data services including ADLS, Azure Key Vault, Azure Kubernetes Service, and Azure Data Factory; CI/CD, Git workflows, code review, and infrastructure as code.
  • Experience with Databricks Unity Catalog or equivalent governance and access-control technology; REST APIs and Kubernetes, with 1+ years of experience in each.
  • Production operations experience including monitoring, on-call or incident response, root cause analysis, Agile principles, strong communication, translating business needs into technical solutions, and influencing without direct authority.
  • Daily experience using AI-assisted development tools with disciplined review of generated code.

Nice to have

  • Databricks or Azure certification.
  • Experience migrating legacy orchestration or ETL to Databricks.
  • Experience with Databricks Auto Loader and Delta Live Tables/Lakeflow Declarative Pipelines for streaming workloads.
  • Experience with event-driven API patterns such as webhooks, asynchronous messaging, and event-carried state transfer.
  • Experience supporting machine learning workloads or MLOps, including real-time or near-real-time feature pipelines.
  • Retail, advertising technology, or retail media domain experience.

Culture & Benefits

  • Work onsite five days per week to support collaboration, alignment, and team connection.
  • Medical, dental, vision, mental health, wellness, AD&D, supplemental insurance, and a health savings account with matching contribution when eligible.
  • 401(k) with Roth option and matching contribution.
  • Five weeks of vacation, seven health and wellness days, three floating holidays, and six company-paid holidays annually.
  • Paid maternity, paternity, and family-care leave.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →