4 дня назад
Principal Data and Platform Engineer (Data Engineering)
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Principal Data and Platform Engineer (Data Engineering): Designing and building cloud-native data platforms, real-time pipelines, and data products using data mesh, data products, data contracts, Databricks, Snowflake, AWS, and GCP with an accent on scalable architecture, streaming systems, and production-grade engineering. Focus on making high-impact architectural decisions, processing large datasets with Spark and Flink, and guiding clients and engineers toward secure, observable, and reliable solutions.
Location: Sydney, New South Wales, Australia; hybrid workplace
Company
is an independent technology consultancy building modern data platforms, intelligent systems, and bespoke software for enterprises, scale-ups, and government clients.
What you will do
- Own the architecture and technical direction of modern, cloud-native data platforms.
- Design and deliver greenfield data platforms, data products, and real-time pipelines using Databricks, Snowflake, AWS, and GCP.
- Build batch and streaming pipelines with Kafka, Flink, Kinesis, Pub/Sub, Airflow, Dataflow, and Glue.
- Process and transform large datasets with Spark and Flink while ensuring production performance.
- Work directly with clients to translate business needs into pragmatic engineering decisions.
- Set engineering standards, review designs and code, mentor engineers, and improve data quality, testing, observability, and operational excellence.
Requirements
- Strong Python and SQL skills with deep experience in Spark and modern data platforms such as Databricks and Snowflake.
- Solid experience with AWS or GCP cloud data services.
- Demonstrated ownership of large-scale data platform architectures, data modelling, and architectural decision-making.
- Experience building and operating large-scale production data pipelines with multiple storage technologies and formats.
- Infrastructure-as-code experience with Terraform or Pulumi and CI/CD experience with GitHub Actions or ArgoCD.
- Experience with data testing and quality frameworks such as dbt, Great Expectations, or Soda, plus consulting or professional services experience.
Culture & Benefits
- Work on high-impact problems and modern platforms rather than legacy systems.
- Collaborate with experienced senior engineers in small, high-calibre teams.
- Autonomy, influence, and opportunities for professional growth.
- Darlinghurst office space with team lunches on Mondays and Wednesdays.
- Training and conference fund.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →