Lead Data Engineer (Databricks)

Location: Hybrid in Coimbatore, India

Company

hirify.global is a growing data product company founded in 2020 that delivers digital solutions primarily for Fortune 500 companies.

What you will do

• Design, develop, maintain, and optimize scalable ETL/ELT pipelines using Databricks, PySpark, and SQL.
• Integrate data from databases, Amazon S3, files, and REST APIs for ingestion and export.
• Build pipelines with Unity Catalog and manage Delta Lake tables using Bronze, Silver, and Gold layers.
• Implement business logic, data transformations, dimensional models, and data warehousing solutions.
• Create, schedule, monitor, and optimize Databricks Jobs and Workflows.
• Ensure data quality and production reliability through validation, error handling, logging, monitoring, and Spark performance tuning.

Requirements

• Strong expertise in Python, PySpark, and advanced SQL.
• Hands-on experience with the Databricks Lakehouse Platform, including Unity Catalog, Delta Lake, Workflows, Jobs, Clusters, Notebooks, and Repos.
• Experience with ETL/ELT development, batch processing, incremental loading, data transformation, and REST API integration.
• Knowledge of dimensional modeling, Star and Snowflake Schemas, Fact and Dimension tables, SCD concepts, and data warehousing.
• Experience with structured and semi-structured data formats including CSV, JSON, Parquet, and Delta.
• Knowledge of partitioning, file optimization, Spark and query performance tuning, Git, and CI/CD best practices.

Nice to have

• Experience with Auto Loader, Spark Declarative Pipelines, Kafka, Airflow, or dbt.
• Databricks certification.

Culture & Benefits

• Competitive salary.
• Strong insurance package.
• Learning and development resources supporting employee growth.