Senior Data Engineer (Python/Spark)

Location: USA (Westport, CT)

Base salary: $130K–$150K per year, with a possible discretionary incentive program and benefits dependent on full-time employment status.

Company

hirify.global delivers first-party data that helps businesses gain insights, activate audiences, and measure impact.

What you will do

• Design, build, and maintain scalable batch and streaming data pipelines using Apache Spark, Kafka, and Flink.
• Build and maintain Bronze, Silver, and Gold medallion layers in the enterprise lakehouse using Delta Lake or Iceberg.
• Develop and optimize PySpark and SQL transformations for large-scale structured, semi-structured, and unstructured datasets.
• Collaborate with data architects to evolve physical and logical data models.
• Implement data quality checks, monitoring, and infrastructure automation using Great Expectations or dbt tests, Terraform, and Helm.
• Troubleshoot pipeline failures, performance bottlenecks, and data incidents; review code and mentor junior and mid-level engineers.

Requirements

• 6+ years of data engineering experience delivering enterprise-scale solutions.
• Expert proficiency in Python and SQL, with required PySpark experience.
• Hands-on experience with Apache Spark, Delta Lake, or Apache Iceberg.
• Experience with Apache Airflow, Prefect, or Dagster, plus cloud data services such as AWS Glue, Azure Data Factory, or GCP Dataflow.
• Proficiency with Git, CI/CD pipelines, Docker, Kubernetes, and dbt.
• Bachelor’s degree in Computer Science, Engineering, or a related technical field.

Nice to have

• Experience with Databricks, Snowflake, or Apache Hudi.
• Knowledge of streaming architectures and Apache Kafka.
• Databricks Certified Data Engineer or AWS Data Analytics Specialty certification.

Culture & Benefits

• Inclusive and accessible work environment focused on respect and equal opportunity.
• Accommodations are available upon request throughout the selection process.
• Medical and other benefits are available based on full-time employment status.
• A discretionary incentive program may be included in the compensation package.