Data Engineer (Cloud Optimization)

TL;DR

Data Engineer (Apache Spark/Databricks): Designing and maintaining end-to-end ETL pipelines for cloud usage data with an accent on scalability and reliability. Focus on building data foundations for product features, analytics, and AI-driven capabilities.

Company

hirify.global is a startup specializing in automated cloud optimization solutions for DevOps and FinOps teams to reduce costs and enhance operational efficiency.

What you will do

• Design, build, and maintain end-to-end ETL pipelines for production use.
• Process large-scale cloud usage data from diverse sources and formats.
• Build data flows using Apache Spark and Databricks.
• Structure data using Data Lake / Lakehouse concepts, including Bronze, Silver, and Gold layers.
• Integrate with external APIs and various data sources.
• Model data to support product features, analytics, and AI-related capabilities.

Requirements

• Experience designing and maintaining reliable ETL pipelines.
• Proficiency with Apache Spark, Databricks, or similar big data technologies.
• Knowledge of Data Lake and Lakehouse architectures.
• Experience integrating with external APIs.
• Ability to ensure data quality, reliability, and observability across pipelines.
• Experience collaborating with backend engineers, data analysts, and product stakeholders.