Data Engineer

TL;DR

Data Engineer: Building and scaling large-scale data pipelines and datasets for the music industry's MusicGraph™ with an accent on data ingestion, optimization, and ownership inference. Focus on designing scalable pipelines, optimizing PostgreSQL and OpenSearch, and collaborating with fullstack teams to integrate complex music data.

What you will do

• Design and maintain scalable data pipelines ingesting multi-source music industry data.
• Build and optimize datasets in PostgreSQL and OpenSearch/Elasticsearch to support matching and clustering algorithms.
• Develop tools for reconciliation, deduplication, ownership inference, and data lineage tracking within the MusicGraph™.
• Collaborate with fullstack engineers to integrate MusicGraph™ features into products.
• Evaluate and tune AWS infrastructure costs and performance.

Requirements

• 3+ years of experience in Data Engineering with large-scale production systems
• Expertise in PostgreSQL database design, optimization, and tuning
• Experience with Elasticsearch or OpenSearch including index design and scaling
• Strong hands-on AWS experience (EC2, OpenSearch, ECS, Glue, Lambda)
• Proficiency in Apache Airflow and Python
• Comfort working with unstructured data, metadata standardization, and complex matching/clustering problems

Nice to have

• Experience designing and building with AI
• Background in music publishing or copyright metadata systems
• Familiarity with music metadata standards (CWR, DDEX ERN, DSR, MWN)
• Experience with large-scale OpenSearch clusters (500M+ documents)
• Strong understanding of data quality, auditing, and versioning in distributed pipelines
• Passion for music and supporting fair creator compensation