5 дней назад
Senior Big Data Engineer (Spark)
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Senior Big Data Engineer (Spark): Design and optimize large-scale data processing pipelines for financial services using Spark, Hadoop, Python, Kafka, and Hive with an accent on scalable data platforms, automation, and reliable deployment. Focus on building high-performance data workflows, integrating CI/CD and DevOps practices, and solving complex scalability and optimization challenges.
Location: Porto, hybrid; candidates must have an existing legal right to work in Europe.
Company
provides technology services for clients in the financial services industry.
What you will do
- Design, develop, and optimize large-scale data processing pipelines with Spark, Hadoop, Hive, and Kafka.
- Write Python scripts for data manipulation, integration, and workflow automation.
- Implement and maintain CI/CD processes and support deployments with XL Release and XL Deploy.
- Collaborate with cross-functional teams to improve system performance, scalability, and reliability.
- Manage source code with Git and automate workflows using CTRL-M.
- Contribute to DevOps practices and continuous improvement.
Requirements
- At least 9 years of experience in big data engineering or a related field.
- Strong experience with Spark, Hadoop, Python, Kafka, SQL, and Linux.
- Background in software development, CI/CD, optimization, and DevOps.
- Experience with Git, XL Release, XL Deploy, and CTRL-M.
- Fluent English at minimum B2 level is required.
- Existing legal right to work in Europe is required.
Nice to have
- Knowledge of French.
- Certifications or experience in advanced data engineering or cloud platforms.
Culture & Benefits
- Hybrid work model in Porto.
- International and distributed working environment.
- Autonomous work with opportunities to share knowledge and contribute to innovative solutions.
Hiring process
- Submit a CV, preferably in English.
- Include consent for the processing and storage of personal data.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →