Apache Spark Developer

Location: 100% remote within the United States

Salary: $125,000–$185,000 annually

Company

hirify.global is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States.

What you will do

• Design, develop, and maintain high-performance distributed data processing applications with Apache Spark.
• Build scalable batch and real-time ETL/ELT pipelines for structured, semi-structured, and streaming data.
• Develop reusable Spark libraries, processing frameworks, and metadata-driven ingestion pipelines using PySpark, Scala, or Spark SQL.
• Optimize Spark jobs for memory utilization, partitioning, shuffle performance, and execution efficiency.
• Deploy Spark workloads on Databricks, EMR, Azure Synapse, or Kubernetes and integrate them with data warehouses, lakehouses, and reporting platforms.
• Support architecture reviews, production troubleshooting, cloud migration, data quality validation, and continuous improvement of enterprise-scale data platforms.

Requirements

• Six or more years of professional software or data engineering experience, including four or more years of hands-on Apache Spark development in enterprise production environments.
• Strong proficiency with PySpark, Scala, or Spark SQL and advanced SQL skills.
• Deep knowledge of Spark architecture, distributed computing, partitioning, shuffling, caching, broadcast joins, and fault tolerance.
• Experience with Hadoop technologies, streaming data, cloud platforms, and lakehouse technologies such as Delta Lake, Apache Iceberg, or Apache Hudi.
• Experience with Git, CI/CD, debugging, performance tuning, data warehousing, dimensional modeling, and Agile Scrum development.
• Applicants must be authorized to work in the United States; new H-1B visa petitions cannot be sponsored.

Nice to have

• Experience with Databricks or Delta Lake lakehouse architectures, workflow orchestration, Spark MLlib, MLflow, or feature engineering pipelines.
• Knowledge of Kubernetes, Docker, data quality frameworks, cloud object storage, Infrastructure as Code, and enterprise monitoring tools.
• Cloud certifications in Azure, AWS, Databricks, or Apache Spark-related technologies.

Culture & Benefits

• Work on cloud-native big data platforms supporting enterprise analytics, AI, business intelligence, financial risk modeling, fraud detection, and healthcare analytics.
• Contribute to real-time streaming analytics processing billions of daily events.
• Collaborate with data architects, data engineers, cloud platform teams, machine learning engineers, and business intelligence developers.
• Full-time direct W2 employment with career growth opportunities.