Data Engineer

TL;DR

Data Engineer (Spark/Databricks): Architect and scale foundational data platforms powering analytics, AI/ML initiatives, and business intelligence with an accent on real-time and batch pipelines, data lakehouse, and enterprise data leverage. Focus on designing scalable ETL workflows, optimizing infrastructure for high-volume data processing, and enabling real-time decision-making across products.

Location: Remote (USA)

Salary: $125,000 - $159,000 per year

Company

Trusted platform for car reviews, research, and shopping tools serving millions of consumers and dealers.

What you will do

• Build and maintain scalable data pipelines for large-scale structured and unstructured data in batch and real-time modes.
• Enhance the data lakehouse central to company decision-making.
• Develop and optimize transactions and pricing pipelines using Spark, Databricks, SQL, Scala, and Python.
• Design process improvements, automate workflows, and redesign infrastructure for scalability.
• Manage AWS-based infrastructure and deploy reliable data systems.
• Collaborate with executive, product, and data teams to resolve technical issues and support data needs.
• Provide technical guidance, troubleshoot pipelines, ensure data quality, and mentor on best practices.

Requirements

• High proficiency in Scala or Python (core codebase languages).
• Fluency in SQL, ETL job development, and large-scale data handling.
• Experience with real-time/streaming data pipelines.
• Familiarity with Spark, Databricks, AWS, Airflow.
• Skills in maintainable software design, OO/functional paradigms, algorithms, and best practices.
• Experience modernizing systems, cloud platforms, CI/CD.
• Strong problem-solving, troubleshooting, and communication in remote/hybrid settings.

Nice to have

• Exposure to AI/ML workflows or interest in ML integration into pipelines.

Culture & Benefits

• Flexible time off and 13 paid holidays.
• Comprehensive health benefits including medical, dental, vision, HSA/FSA contributions.
• 401k with 100% match up to 6%, stock purchase, pet insurance.
• Up to 4 months paid parental leave, sabbatical, education reimbursement.
• Well-being sessions, FitCash for gym, HeartCash for donations, social impact PTO.
• Ongoing career development and employee recognition programs.