Lead Consultant, Data Engineer (Databricks)

TL;DR

Lead Consultant, Data Engineer (Databricks): Designing and implementing complex ETL/ELT pipelines and data warehousing solutions for enterprise clients with an accent on Databricks, AWS, and Azure integrations. Focus on SAP data structures, real-time data processing, and establishing robust CI/CD strategies for data platforms.

Location: Remote (Must be based in the US)

Company

Databricks-focused data and AI consulting firm specializing in artificial intelligence, data, and analytics solutions for high-profile enterprise clients.

What you will do

• Lead individual work streams and execute client-facing data engineering projects independently.
• Design and implement complex ETL/ELT pipelines to optimize data processing times.
• Lead small-scale data warehousing projects with a focus on measurable performance enhancements.
• Develop real-time data processing and managed streaming solutions.
• Implement security, compliance measures, and CI/CD version control strategies.
• Interpret and integrate SAP data at the table level within the context of business processes.

Requirements

• B.S. in Computer Science or equivalent.
• 3-5 years of experience in data engineering and big data.
• At least 2 years of experience working directly with external stakeholders and clients.
• Extensive hands-on experience deploying pipelines using Databricks (Must).
• Must be based in the US; no work sponsorship is available.
• Experience with SAP S/4HANA or SAP HANA data structures.

Culture & Benefits

• Opportunity to work with diverse clients ranging from Fortune 100 companies to small businesses.
• Collaborative environment where challenging the status quo and developing new methodologies is encouraged.
• Access to experienced data engineers for rapid skill expansion and professional growth.
• Inclusive and diverse team culture consisting of entrepreneurial minds and data experts.