6 дней назад
Lead Big Data Engineer (Azure)
Вакансия из Hirify Global, списка международных tech-компанийДля мэтча и отклика нужен Plus
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Lead Big Data Engineer (Azure) (AWS/Big Data): Supporting and enhancing a cloud-based data platform and ingestion pipelines for regulated healthcare data with an accent on large-scale processing, reliability, and data quality. Focus on optimizing Spark workflows, orchestrating pipelines, strengthening deployment infrastructure, and maintaining security and compliance.
Location: Remote, Chile; includes readiness to travel to the United States for a three-month business trip in Q1 2027.
Company
operates in the engineering and technology domain.
What you will do
- Support, maintain, and enhance an AWS-based data platform and data ingestion pipelines.
- Develop and optimize data processing workflows with Python, PySpark, Apache Spark, and AWS Glue.
- Monitor and troubleshoot pipelines to ensure reliability, performance, and data quality.
- Work with AWS services including S3, Aurora PostgreSQL, Athena, Lambda, SQS, SNS, and EventBridge.
- Maintain workflow orchestration with Apache Airflow and support infrastructure and deployments using Docker, Terraform, and Amazon EKS.
- Apply automated testing, code quality checks, CI/CD, AI-assisted development tools, and data security and compliance practices while collaborating with distributed engineering teams.
Requirements
- 5+ years of experience in Big Data or Data Engineering, including production data platforms.
- Strong proficiency in Python, PySpark, and Apache Spark for large-scale data processing.
- Hands-on experience with AWS Glue, S3, Athena, Aurora PostgreSQL, and Lambda.
- Experience with Apache Airflow for data pipeline orchestration and scheduling.
- Strong knowledge of SQL, data ingestion, ETL processes, and data quality practices.
- Experience with Docker, Terraform, and CI/CD pipelines using GitHub Actions or similar tools.
- Familiarity with Python development and testing tools such as pytest, Pydantic, ruff, or mypy.
- Readiness to travel to the United States for a three-month business trip in Q1 2027.
- Upper-intermediate or higher English, strong communication skills, and problem-solving skills.
Nice to have
- Understanding of healthcare data regulations such as HIPAA/HITRUST, multi-tenant architectures, or data retention requirements.
Culture & Benefits
- Remote work arrangement.
- Collaboration with a distributed engineering team.
- Work on healthcare data in a regulated environment.
Hiring process
- Complete the application form; the recruiting team will follow up.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →
Музыка
Осень
Поддержка