обновлено 12 дней назад
PySpark Developer
Вакансия из Hirify Global, списка международных tech-компанийДля мэтча и отклика нужен Plus
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
PySpark Developer (Python/Apache Spark): Developing innovative data solutions and supporting operations in the investment banking and financial services domain with an accent on Spark architecture, data processing, and query optimization. Focus on designing scalable data workflows, optimizing complex SQL queries, working with cloud platforms and modern storage formats, and ensuring software quality through testing and CI/CD.
Location: Porto, Portugal — hybrid
Company
Technology hub in Portugal developing innovative solutions and supporting operations for the investment banking and financial services business.
What you will do
- Develop data solutions using Python and the PySpark ecosystem.
- Work with Apache Spark architecture, including drivers, workers, DAGs, lazy evaluation, and caching.
- Optimize complex SQL queries and data processing workflows.
- Work with Parquet, Delta Lake, and Avro storage formats in cloud environments.
- Use Databricks or EMR, Git, CI/CD pipelines, and workflow orchestration tools.
- Apply unit testing and Spark data mocking to maintain software quality.
Requirements
- More than 9 years of professional experience.
- At least 5 years of software development or data engineering experience, with strong specialization in PySpark and Apache Spark.
- Advanced Python expertise and strong SQL proficiency.
- Experience with AWS, Azure, or GCP and platforms such as Databricks or EMR.
- Experience with Git, CI/CD pipelines, and a workflow orchestrator such as Apache Airflow or Prefect.
- Fluent English required.
Culture & Benefits
- Health insurance.
- Personalized training plan with a dedicated training budget.
- Regular feedback and professional growth support.
- Remote onboarding process.
- Team events every semester.
- Culture of proximity and transparency.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →