Senior Data Platform Engineer (AI)
ΠΡΡΡ & Π‘ΠΎΠΏΡΠΎΠ²ΠΎΠ΄
ΠΠ»Ρ ΠΌΡΡΡΠ° Ρ ΡΡΠΎΠΉ Π²Π°ΠΊΠ°Π½ΡΠΈΠ΅ΠΉ Π½ΡΠΆΠ΅Π½ Plus
ΠΠΏΠΈΡΠ°Π½ΠΈΠ΅ Π²Π°ΠΊΠ°Π½ΡΠΈΠΈ
TL;DR
Senior Data Platform Engineer (GCP/AI): Implementing and maintaining global data infrastructure, including relational, vector, and graph databases, with an accent on building an enterprise-scale data layer for agentic AI. Focus on developing robust ETL/ELT pipelines, ensuring data integrity across distributed systems, and optimizing cloud-scale data lakes on GCP.
Location: Hybrid in Cape Town, South Africa
Company
helps marketing and creative teams in top brands scale their creative production and automate campaigns through an AI-powered platform.
What you will do
- Implement and configure database technologies including relational, vector, and graph databases.
- Design and maintain scalable data lake architectures on Google Cloud Platform, specifically utilizing BigQuery.
- Develop and optimize robust ETL/ELT pipelines using Apache Airflow, dbt, and Google Dataflow.
- Implement validation and reconciliation mechanisms to ensure data integrity and consistency across distributed stores.
- Build automated data quality checks, monitoring, and alerting systems.
- Collaborate with product and engineering teams to define data requirements and technology selection.
Requirements
- Hands-on experience with relational database schema design, query optimization, and operational management.
- Practical production experience with vector databases (e.g., Pinecone, Weaviate, pgvector, Vertex AI Search).
- Working knowledge of graph database technologies such as Neo4j or Spanner Graph.
- Strong experience with the GCP data ecosystem, including BigQuery, Cloud Dataflow, Cloud Composer, and Pub/Sub.
- Proven ability to build reliable ETL/ELT pipelines using Airflow and dbt.
- Deep understanding of data integrity and consistency challenges in distributed systems.
Culture & Benefits
- Strong commitment to diversity, equity, and inclusion for all applicants.
- Access to various Employee Resource Groups (ERGs) for community support and networking.
- Collaborative work environment that values curiosity and problem-solving.
- Emphasis on well-being and taking time to look after employees and the planet.
ΠΡΠ΄ΡΡΠ΅ ΠΎΡΡΠΎΡΠΎΠΆΠ½Ρ: Π΅ΡΠ»ΠΈ ΡΠ°Π±ΠΎΡΠΎΠ΄Π°ΡΠ΅Π»Ρ ΠΏΡΠΎΡΠΈΡ Π²ΠΎΠΉΡΠΈ Π² ΠΈΡ ΡΠΈΡΡΠ΅ΠΌΡ, ΠΈΡΠΏΠΎΠ»ΡΠ·ΡΡ iCloud/Google, ΠΏΡΠΈΡΠ»Π°ΡΡ ΠΊΠΎΠ΄/ΠΏΠ°ΡΠΎΠ»Ρ, Π·Π°ΠΏΡΡΡΠΈΡΡ ΠΊΠΎΠ΄/ΠΠ, Π½Π΅ Π΄Π΅Π»Π°ΠΉΡΠ΅ ΡΡΠΎΠ³ΠΎ - ΡΡΠΎ ΠΌΠΎΡΠ΅Π½Π½ΠΈΠΊΠΈ. ΠΠ±ΡΠ·Π°ΡΠ΅Π»ΡΠ½ΠΎ ΠΆΠΌΠΈΡΠ΅ "ΠΠΎΠΆΠ°Π»ΠΎΠ²Π°ΡΡΡΡ" ΠΈΠ»ΠΈ ΠΏΠΈΡΠΈΡΠ΅ Π² ΠΏΠΎΠ΄Π΄Π΅ΡΠΆΠΊΡ. ΠΠΎΠ΄ΡΠΎΠ±Π½Π΅Π΅ Π² Π³Π°ΠΉΠ΄Π΅ β