3 дня назад
Senior Data Engineers
Вакансия из Hirify Global, списка международных tech-компанийДля мэтча и отклика нужен Plus
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Senior Data Engineers (Kafka/Spark/Flink): Building and operating real-time data ingestion systems and layered data warehouses for business logs, user behavior, and database data with an accent on streaming ETL, data quality, and pipeline stability. Focus on optimizing Kafka and Spark/Flink workloads, resolving latency and reliability issues, and delivering dependable data for dashboards, services, and user analytics.
Location: Singapore, Singapore
Company
develops a Smart TV operating system and connected digital platform used by millions of households.
What you will do
- Own the full lifecycle of real-time data ingestion for business logs, user behavior data, and operational database data.
- Design and optimize layered real-time data warehouse architectures, streaming ETL pipelines, metrics, dashboards, and monitoring data.
- Maintain Kafka pipelines and resolve message backlog, data skew, data loss, duplication, availability, and stability issues.
- Develop and operate Spark and Flink real-time tasks while improving resource utilization, latency, throughput, and reliability.
- Collaborate with business teams, analysts, and data product managers to decompose requirements, implement solutions, verify releases, and iterate on data products.
- Improve data standards, quality governance, pipeline redundancy, and downstream data services for BI, user profiles, and business effectiveness measurement.
Requirements
- Bachelor’s degree or above in Computer Science, Software Engineering, Data Science, or a related field, plus 3+ years of professional big data development experience.
- Experience implementing real-time data warehouses and end-to-end data ingestion systems, including multi-source processing, CDC synchronization, data cleaning, and exception handling.
- Expert knowledge of Kafka and hands-on experience with Spark, Spark SQL, Flink, and Flink SQL.
- Experience with BigQuery, DBT, Databricks, cloud data warehouse modeling, pipeline orchestration, metric iteration, and dashboard development.
- Proficiency in at least one of Python, Scala, or Java, plus Linux commands, Shell scripting, relational databases, Binlog/CDC, data quality monitoring, SLA governance, and incident reviews.
- English proficiency in reading, writing, and verbal communication is required for collaboration with overseas teams and business stakeholders.
Nice to have
- Overseas study or work experience.
- Experience with Google Cloud, Azure, or other mainstream cloud big data platforms.
- Experience collaborating with overseas teams and governing real-time data systems.
Culture & Benefits
- Work within the R&D department on a Smart TV OS and connected digital platform.
- Collaborate with international teams and business stakeholders.
- Work in an environment emphasizing safety, security, innovation, and entrepreneurial initiative.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →