4 дня назад
Senior Data Engineer (Python/SQL)
Вакансия из Hirify Global, списка международных tech-компанийДля мэтча и отклика нужен Plus
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Senior Data Engineer (Python/SQL): Designing and building scalable batch and streaming data pipelines, transformation layers, and data models for an enterprise lakehouse with an accent on Apache Spark, Kafka, Flink, and medallion architecture. Focus on optimizing large-scale PySpark and SQL transformations, implementing data quality monitoring and infrastructure as code, and mentoring data engineers.
Location: Remote in Hungary, Debrecen
Company
provides an inclusive and accessible environment for employees and customers.
What you will do
- Design and implement scalable batch and streaming data pipelines using Apache Spark, Kafka, and Flink.
- Build and maintain Bronze, Silver, and Gold medallion layers in the lakehouse using Delta Lake or Apache Iceberg.
- Develop and optimize complex SQL and PySpark transformations for large-scale structured, semi-structured, and unstructured datasets.
- Collaborate with data architects to evolve physical and logical data models.
- Implement data quality checks and monitoring with Great Expectations or dbt tests.
- Write infrastructure as code, participate in code reviews, troubleshoot pipeline incidents, and mentor junior and mid-level data engineers.
Requirements
- 6+ years of data engineering experience delivering enterprise-scale solutions.
- Expert proficiency in Python and SQL, with required PySpark experience.
- Hands-on experience with Apache Spark, Delta Lake, or Apache Iceberg.
- Experience with Apache Airflow, Prefect, or Dagster, plus cloud data services such as AWS Glue, Azure Data Factory, or GCP Dataflow.
- Proficiency with Git, CI/CD pipelines, Docker, Kubernetes, Terraform, Helm, and dbt.
- Bachelor’s degree in Computer Science, Engineering, or a related technical field.
Nice to have
- Experience with Databricks, Snowflake, or Apache Hudi.
- Knowledge of streaming architectures and Apache Kafka.
- Databricks Certified Data Engineer or AWS Data Analytics Specialty certification.
Culture & Benefits
- Full-time employment in an inclusive and accessible work environment.
- Accommodations are available by request throughout the selection process.
- Commitment to a diverse workforce and inclusive workplace culture.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →