Lead DataOps Engineer (Azure)

TL;DR

Lead DataOps Engineer (Azure/Databricks): Leading the design and implementation of DataOps frameworks and automation for insurance data products with an accent on pipeline orchestration and system observability. Focus on operationalizing data products end-to-end, driving CI/CD adoption, and ensuring technical quality for analytics and AI use cases.

Location: Hybrid (Short Hills, NJ or Charlotte, NC)

Salary: $130,000 - $176,000 / Yr

Company

Selective is a mid-sized U.S. domestic property and casualty insurance company with nearly 100 years of consistent financial performance.

What you will do

• Lead a high-performing team in delivering key data components and business objectives.
• Design and implement organization-wide DataOps frameworks, standards, and best practices.
• Operationalize end-to-end data products in collaboration with Architecture, Analytics, and AI/ML teams.
• Drive the adoption of automation, orchestration (Airflow, ADF, Dagster, dbt), and observability for data pipelines.
• Establish reliability targets and change management processes for data contracts.
• Mentor team members and conduct rigorous design, code, and test plan reviews.

Requirements

• 8+ years of experience in data engineering, analytics engineering, or platform engineering.
• Proficiency with cloud data platforms (Azure, AWS, or GCP) and Databricks/lakehouse architecture.
• Expertise in PySpark, SQL, and modern data modeling.
• Proficiency with CI/CD tools such as Azure DevOps, GitHub Actions, or GitLab.
• Proven track record of taking accountability for leading others to complete complex deliverables.
• Must be based in or able to work from Short Hills, NJ or Charlotte, NC.

Nice to have

• Experience supporting machine learning or AI workloads in production.
• Familiarity with streaming platforms like Kafka, Event Hubs, or Kinesis.
• Property Casualty Insurance industry experience.
• SAFe Agile work experience and a degree in Computer Science or MIS.

Culture & Benefits

• Comprehensive health care plans and retirement savings plan with company match.
• Discounted Employee Stock Purchase Program.
• Tuition assistance and reimbursement programs.
• 20 days of paid time off.
• Incentive plan eligibility across all levels.