8 дней назад
Senior Data Engineer (Databricks)
84 000 - 128 667$
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Senior Data Engineer (Databricks): Building and operating production data pipelines and curated data products on an Azure Databricks lakehouse with an accent on medallion architecture, Delta Lake, data contracts, and data quality. Focus on designing reliable ingestion, developing dimensional and ML datasets, orchestrating Dagster assets, and solving production reliability, governance, performance, and cost challenges.
Location: Remote within the United States, with the listed location as Birmingham, Alabama / Nebraska
Salary: $84,000–$128,667 per year, plus potential annual incentive compensation.
Company
provides insurance and financial protection products that help customers manage life’s uncertainties.
What you will do
- Design, build, and operate production data pipelines on an Azure Databricks lakehouse using Python, SQL, Apache Spark, and Delta Lake.
- Implement Bronze, Silver, and Gold data flows, including ingestion from APIs, databases, files, cloud storage, and SaaS platforms.
- Develop trusted dimensional models, slowly changing dimensions, fact and bridge tables, aggregates, serving tables, and reproducible ML feature and training datasets.
- Author Open Data Contract Standard contracts and implement automated data-quality checks for uniqueness, validity, referential integrity, freshness, and business rules.
- Orchestrate and operate data assets with Dagster Cloud, including schedules, sensors, dependencies, backfills, monitoring, and on-call recovery.
- Maintain governance, CI/CD, observability, documentation, and performance optimization across development, branch, and production deployments.
Requirements
- Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent practical experience.
- 3+ years of experience building and supporting production data pipelines in a cloud data platform environment.
- Strong hands-on Python and SQL skills, with experience using Databricks or a comparable Spark-based lakehouse and Delta Lake.
- Experience with medallion or multi-layer lakehouse design, incremental loading, state management, dimensional modeling, ELT, and data quality practices.
- Experience with orchestration, Git-based source control, pull requests, automated testing, CI/CD, pipeline monitoring, alerting, and production troubleshooting.
- Must be eligible to work remotely within the United States as indicated by the listed Birmingham, Alabama / Nebraska location.
Nice to have
- Databricks Data Engineer certification or equivalent expertise.
- Experience with Unity Catalog, dbt on Databricks, Python-based Delta Lake modeling, and Type 2 history.
- Experience with dlt, Airbyte, Meltano, Fivetran, or Dagster assets, checks, sensors, schedules, and branch deployments.
Culture & Benefits
- Full-time employee position with health, dental, and vision insurance.
- Paid time off, paid parental leave, short-term disability, and a cultural observance day.
- Mental health benefits and an employee assistance program.
- Pension contributions and a 401(k) plan with company matching.
- Annual incentive opportunities based on individual and company performance.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →
Похожие вакансии
9 дней назад
Databricks Data Engineer (Risk Technology)
120 000 - 160 000$
12 дней назад
Expert Data Engineer (Databricks)
12 дней назад
Data Engineer (Databricks)
110 000 - 126 000$
12 дней назад
Senior Data Engineer (Databricks/Snowflake)
12 дней назад
Senior Data Engineer (Python/PySpark, Databricks)
154 627 - 165 000$
11 дней назад
Databricks Platform Engineer (Azure)
110 000 - 125 000$