2 дня назад
Senior Data Engineer (Apache Airflow)
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Senior Data Engineer (Apache Airflow/AWS): Building and operating reliable data pipelines for a healthcare provider data platform with an accent on Apache Airflow modernization, data quality, observability, and scalable production operations. Focus on designing reusable workflows, optimizing Spark-based ETL on AWS, and solving complex reliability, cost, and data consistency challenges.
Location: Remote from Armenia, Bulgaria, Colombia, Cyprus, Georgia, Kazakhstan, Latvia, Mexico, Poland, LATAM, Romania, Serbia, or Ukraine
Company
is a global software engineering company delivering data, analytics, and AI solutions for clients across healthcare and other industries.
What you will do
- Own the end-to-end design, deployment, and operation of production data pipelines.
- Modernize a large Apache Airflow environment and develop reusable DAGs, libraries, and engineering standards.
- Build workflows for data processing, quality checks, provider updates, and data publishing.
- Develop data-quality frameworks, data contracts, source-of-truth rules, monitoring, anomaly detection, and alerting.
- Maintain and optimize Spark-based ETL jobs running on AWS EMR, S3, and Elasticsearch.
- Collaborate with product, engineering, operations, and data teams on architecture, releases, technical planning, and production troubleshooting.
Requirements
- 5+ years of software or data engineering experience with production data systems.
- Strong Python, SQL, Apache Airflow, AWS, and production data pipeline experience.
- Experience with Apache Spark or similar distributed processing frameworks, large datasets, Parquet, and data lakes.
- Experience with data validation, reconciliation, monitoring, software design, testing, Git, CI/CD, and production releases.
- Ability to balance scalability, reliability, maintainability, delivery speed, and cost while working independently.
- Strong communication skills and willingness to mentor engineers and contribute to technical decisions.
Nice to have
- Healthcare or provider data experience, Elasticsearch or OpenSearch, Scala or Groovy, and BigQuery.
- Docker, Terraform, CloudFormation, AWS CDK, data observability, lineage, or data contracts.
- Selenium, browser automation, web data collection, or integrations with Microsoft Graph, SharePoint, Slack, or Teams.
- pytest, Ruff, mypy, or experience with HIPAA and other compliance-sensitive environments.
Culture & Benefits
- Remote, hybrid, and office work formats are available according to the role's location and contract conditions.
- Vacation, sick pay, health insurance support, and time off for state holidays are provided according to regional conditions.
- IT certification costs, professional courses, mentoring, and learning platforms are available.
- Corporate events, technical and everyday workplace support, mental health programs, and flexible development opportunities are offered.
Hiring process
- CV review followed by an HR interview.
- Communication and technical assessments, including evaluation of English and relevant technologies.
- Meeting with the project team and possible client interview preparation.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →