Senior Data Engineer (Databricks)

Location: Remote within the United States; applicants must be authorized to work for any employer in the United States.

Salary: $135,000–$160,000 annually, with possible annual bonus and benefits depending on the position.

Company

hirify.global delivers digital learning experiences and education products for learners and educators.

What you will do

• Define learning events and event standards with product platform teams, and build reliable pipelines connecting learning interactions to actionable insights.
• Design and operate Databricks lakehouse solutions on AWS using Delta Lake, medallion architecture, Unity Catalog, and scalable data models.
• Develop high-volume parallel and distributed ETL/ELT pipelines with Apache Spark, PySpark, Scala, and Databricks.
• Build real-time and near-real-time event-driven ingestion, processing, and transformation pipelines.
• Establish end-to-end data quality through schema validation, monitoring, alerting, and observability.
• Maintain Databricks Workflows, optimize pipeline performance, document architectures and data flows, and deliver solutions across the software development lifecycle in collaboration with product, engineering, analytics, and business stakeholders.

Requirements

• 5+ years of experience in Data Engineering.
• Deep expertise in event-driven data and lakehouse architecture, including product event design, event schemas, real-time or near-real-time processing, Delta Lake, medallion patterns, Unity Catalog, Databricks, and cloud-native data solutions.
• Hands-on experience with Databricks Delta Lake, Delta Live Tables, Databricks Workflows, Unity Catalog, Databricks SQL, and MLflow.
• Experience designing event-driven architectures, event ingestion, streaming pipelines, and real-time or near-real-time processing with Apache Kafka, cloud-native streaming services, or equivalent platforms.
• Experience with AWS services including S3, Redshift, Glue, Lambda, EMR, Athena with Iceberg, Step Functions, and IAM, integrated with Databricks where applicable.
• Proficiency in Python with PySpark, Scala with Spark, or SQL for pipeline development, event processing, and data transformation.
• 3+ years of experience working with cloud platforms, primarily AWS, including Databricks environment architecture and operations, workspace configuration, cluster policies, instance profiles, security, and cost optimization.
• 1+ years of experience with workflow automation and pipeline orchestration using Databricks Workflows, Apache Airflow with the Databricks provider, or equivalent tools.
• Experience collaborating with product platform teams on event requirements, schemas, instrumentation, reliable pipelines, curated data products, learning insights, and end-to-end event data quality.
• Experience optimizing high-volume event processing and distributed data pipelines with Spark, Delta Lake, and cloud-native technologies.

Nice to have

• Experience in the EdTech domain.
• Experience with infrastructure-as-code, CI/CD, Databricks Repos or Git folders, Jira, and Agile/Kanban delivery frameworks.

Culture & Benefits

• Remote work within the United States.
• Work focused on digital learning experiences and the future of education.
• Culture emphasizing innovation, teamwork, career growth, and personal well-being.
• Possible annual bonus and medical and other benefits depending on the position.