SDE III - Data Engineering (AI)

TL;DR

SDE III - Data Engineering (AI): Building and scaling the intelligence backbone for a generative AI-powered commerce platform with an accent on large-scale data pipelines, identity resolution, and feature stores. Focus on designing reliable ETL/ELT frameworks, optimizing distributed data processing, and developing AI-powered engineering tools.

Company

An AI commerce platform transforming shopping experiences through inspiration-led discovery, backed by Google and InMobi Group.

What you will do

• Design and build scalable batch and real-time data pipelines using Spark, Flink, Kafka, and Airflow.
• Develop systems for user identity resolution and cross-surface signal aggregation across Mobile, TV, and OEM ecosystems.
• Build ingestion and enrichment pipelines for affiliate feeds, merchant catalogs, and product metadata.
• Create reusable feature generation frameworks and low-latency pipelines for ML training and online inference.
• Develop internal AI-powered tools and agents for pipeline debugging, SQL optimization, and metadata discovery.
• Own production services and pipelines with strong SLAs, ensuring observability through monitoring, lineage, and quality checks.

Requirements

• 6–10 years of experience in Data Engineering, Distributed Systems, or Data Platform development.
• Strong hands-on experience with Apache Spark, Kafka, Flink, and Airflow.
• Proficiency in Python, SQL, and either Scala or Java.
• Strong understanding of dimensional modeling, data warehousing, and large-scale schema design.
• Experience with GCP, Databricks, BigQuery, and Infrastructure as Code.
• Proven ability to own large-scale production systems end-to-end.

Nice to have

• Experience with product catalogs, affiliate commerce platforms, and recommendation systems.
• Expertise in identity resolution, audience platforms, and Customer 360 systems.
• Experience building feature stores and MLOps infrastructure.
• Exposure to LLM-powered developer tools, agents, and copilots.

Culture & Benefits

• Work on a first-of-its-kind generative AI commerce platform impacting billions of devices.
• Collaborative environment partnering with Applied Scientists, ML Engineers, and Product Managers.
• High engineering standards with a focus on scalability, reliability, and operational excellence.
• Opportunity to drive technical leadership and mentor junior engineers.