Senior Data Engineer (Databricks)

TL;DR

Senior Data Engineer (Databricks): Designing and optimizing complex data pipelines and Lakehouse architectures for a life sciences data platform with an accent on scalability, data integrity, and high-performance workflows. Focus on building advanced AI-driven solutions, leading architectural discussions, and collaborating with cross-functional teams across US and European time zones.

Location: Remote. Requires online presence from 9 AM to 5 PM EST.

Company

hirify.global is a technology services company specialized in building advanced data platforms and AI-driven solutions for industry leaders.

What you will do

• Design, develop, and optimize data pipelines and workflows within the Databricks ecosystem.
• Lead architecture discussions with engineering, product, and data science teams to implement advanced analytics solutions.
• Build and fine-tune Databricks workflows to ensure high performance and reliability.
• Implement scalable data ingestion processes from diverse sources.
• Ensure data integrity, accuracy, and security throughout all processing stages.
• Participate in daily Scrum ceremonies and maintain effective collaboration in a distributed team environment.

Requirements

• 10+ years of experience in software development, with a focus on data engineering or data warehousing.
• 5+ years of hands-on experience specifically with the Databricks ecosystem.
• Expert proficiency in Python and TypeScript.
• Deep understanding and practical experience with Lakehouse architecture, Spark, Glue, and Delta tables/Iceberg.
• Experience with cloud data storage and processing, particularly AWS services (S3, Step Functions, Lambda, Airflow).
• Must be able to maintain online availability from 9 AM to 5 PM EST.

Nice to have

• Databricks certifications.
• AWS or Azure DevOps/Solutions Architect certifications.
• Knowledge of DevOps and MLOps principles.
• Experience collaborating directly with Data Scientists and ML developers.
• Previous leadership or management experience in technology services companies.

Culture & Benefits

• Opportunity to work on flagship projects for life sciences industry leaders like Pfizer and Moderna.
• Technical leadership role in a high-growth environment focused on AI and data-driven research.
• Collaborative, distributed team environment working with global engineering talent.
• Engagement with sophisticated architectural challenges in the rapidly evolving AI/Data space.