9 дней назад
Sr. Databricks Data Engineer (Databricks)
170 000 - 185 000$
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Sr. Databricks Data Engineer (Databricks): Designing, building, migrating, and operating secure, scalable data pipelines in a Databricks Lakehouse for a U.S. Department of Transportation data modernization effort with an accent on legacy-source migration, metadata-driven ingestion, Medallion Architecture, and data governance. Focus on developing Python, PySpark, and SQL workloads, optimizing Delta Lake pipelines, implementing reconciliation and quality controls, and supporting AWS/OneDOT production operations.
Location: McLean, VA, USA
Salary: $170,000–$185,000 per year
Company
delivers digital services, advanced analytics, AI/ML, cybersecurity, and other technology solutions to U.S. government agencies.
What you will do
- Design and implement reusable ingestion and transformation patterns for legacy and external data sources in Databricks.
- Build and operate Bronze, Silver, and Gold pipelines with batch and incremental loads, orchestration, monitoring, alerting, retries, and production support.
- Support data platform migration to the AWS/OneDOT environment.
- Develop and optimize Python, PySpark, and SQL workloads using Databricks, Delta Lake, Unity Catalog, Auto Loader or Lakeflow, and Databricks Jobs/Workflows.
- Implement source-to-target reconciliation, data validation, schema evolution, lineage, access controls, documentation, and engineering standards.
- Collaborate with project management, architecture, business, BI, cloud/DevSecOps, security, and governance stakeholders.
Requirements
- Bachelor’s degree in computer science, data engineering, information technology, engineering, or a related field; equivalent experience may be considered.
- At least 7 years of professional data engineering experience, including hands-on Databricks/Spark and production pipeline experience.
- Advanced Python, PySpark, and SQL skills, with experience in Databricks, Delta Lake, AWS/S3, CSV/JSON ingestion, REST APIs, Git/CI/CD, and orchestration.
- Experience securely handling PII and sensitive data, including access controls and secrets management.
- Must meet applicable DOT contract security and suitability requirements.
Nice to have
- Databricks or AWS data engineering certification.
- Experience modernizing Sybase, SAP IQ, or similar legacy platforms into Databricks.
- Experience with Unity Catalog, metadata-driven ingestion, Auto Loader/Lakeflow, Databricks Jobs/Workflows, lineage tools, and scalable ingestion frameworks.
- Federal or DOT delivery experience, Agile/Kanban, production O&M, cloud cost optimization, data governance, streaming, or AI/ML data pipelines.
Culture & Benefits
- Medical, dental, vision, life, short-term disability, and long-term disability insurance.
- 11 federal holidays and paid time off.
- 401(k) with company matching.
- Flexible Spending Accounts for commuter, medical, and dependent care expenses.
- Tuition assistance and charitable contribution matching.
- Eligible employees may receive performance-based incentives.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →
Похожие вакансии
Databricks
10 дней назад
Tax Data & Technology Manager (AI)
161 000 - 221 450$
10 дней назад
Data Engineer (Databricks)
110 000 - 126 000$
Databricks
13 дней назад
Specialist Solutions Architect - Data Engineering & Warehousing (Digital Native Business) (Data Engineering)
180 000 - 247 500$
13 дней назад
Databricks SME (Remote) (Azure)
175 000 - 195 000$
1 день назад
Associate Data Engineer (Databricks)
85 000 - 115 000$
Databricks
14 дней назад
Senior Solutions Architect (Financial Services)
219 100 - 301 300$