8 дней назад
AWS Lakehouse Data Engineer
Вакансия из Hirify Global, списка международных tech-компанийДля мэтча и отклика нужен Plus
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
AWS Lakehouse Data Engineer (AWS/Apache Iceberg): Building and operating scalable AWS-native lakehouse platforms and batch and streaming data pipelines with an accent on Python, PySpark, Apache Iceberg, governance, and data quality. Focus on implementing ACID transactions, schema evolution, lineage, CI/CD, infrastructure as code, and performance and cost optimization across distributed data workloads.
Location: Remote, United States; the posting lists McLean, Virginia, United States and specifies remote offsite work.
Company
delivers outsourced services and workforce solutions across North America.
What you will do
- Build and operate batch and streaming data pipelines from APIs, databases, files, event streams, and external partners.
- Develop production ETL/ELT pipelines with Python and PySpark for analytics, reporting, visualization, and machine learning.
- Design an AWS-native Delta Lakehouse-style platform on Amazon S3 using Apache Iceberg and Apache Parquet.
- Implement ACID transactions, schema evolution, snapshot isolation, time travel, querying, and performance optimization.
- Establish data governance, metadata cataloging, lineage, access control, data quality checks, and audit controls.
- Automate AWS provisioning, CI/CD, testing, environment promotion, monitoring, and operational documentation.
Requirements
- Bachelor’s degree in a relevant engineering, information technology, computer science, or data engineering field, or four years of equivalent practical experience.
- Six years of relevant experience building AWS-native data lake or lakehouse architectures on Amazon S3.
- Strong production experience with Python, PySpark, ETL/ELT, data modeling, transformation, and performance tuning.
- Hands-on Apache Iceberg experience, including ACID transactions, schema evolution, time travel, and query optimization.
- Advanced SQL skills and experience with AWS data governance, cataloging, lineage, access control, IAM, KMS, and security fundamentals.
- Experience with infrastructure as code, CI/CD data workflows, distributed workload troubleshooting, cost management, and cross-functional collaboration.
Nice to have
- Experience with Databricks, Delta Lake, AWS Step Functions, MWAA, Kinesis, DMS, Lambda, or MSK.
- Experience with Terraform, CloudFormation, Jenkins, CodePipeline, GitHub Actions, Docker, or similar DevOps tools.
- Knowledge of Amazon QuickSight, Tableau, Power BI, AI-assisted coding tools, graph modeling, ontology, taxonomy, entity resolution, or hybrid retrieval.
Culture & Benefits
- Remote offsite work model.
- Cross-functional collaboration and emphasis on high-quality engineering documentation.
- Eligible employees may receive medical, dental, vision, spending account, life insurance, and voluntary plan benefits.
- Eligible employees may participate in a 401(k) plan.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →