Data Engineer III (Spark)

Location: 811 Excellence Dr., Bentonville, AR 72716, United States

Salary: $110,302.00–$180,000.00 per year, plus annual or quarterly performance incentives.

Company

hirify.global is Walmart's advertising technology organization.

What you will do

• Analyze business problems and requirements using analytics, big data analytics, and automation.
• Extract, transform, validate, and model data from databases and cloud systems.
• Design and develop scalable distributed data pipelines, ETL solutions, data warehouses, data marts, and efficient data flows.
• Analyze data dependencies, relationships, system integration challenges, and data quality issues.
• Write, test, document, and deploy code and application features; create proofs of concept, playbooks, and training documentation.
• Support data governance, troubleshoot systems, oversee less experienced programmers, and provide progress updates and expert guidance.

Requirements

• Master’s degree or equivalent in Computer Science, Engineering, or a related field; or a bachelor’s degree or equivalent in one of these fields plus 2 years of software engineering or related experience.
• Experience analyzing business requirements, performing data analysis, and developing technical design documents with product and data teams.
• Experience designing and building scalable distributed data pipelines and ETL solutions using Spark, Hadoop, Hive, Azure Databricks, Azure Data Factory, Azure Blob Storage, GCP Dataproc, and BigQuery.
• Experience developing data solutions with object-oriented programming in Python or Scala, automating workflows with Python and shell scripting, and building lightweight ETL frameworks.
• Experience optimizing Spark Core, Spark SQL, and Spark Streaming applications; implementing pipelines on Microsoft Azure, Databricks, Google Cloud Platform, and BigQuery.
• Experience orchestrating workflows with Apache Airflow, Azure Data Factory, GCP Cloud Composer, Automic, or Oozie.
• Experience establishing CI/CD pipelines with GitHub, Jenkins, JFrog, or Artifactory and managing code with Git-based version control.
• Experience writing optimized SQL for data extraction, transformation, and warehousing, including Hive and BigQuery, and designing data models across RDBMS and cloud systems.
• Experience collaborating with cross-functional teams and applying Agile and Waterfall methodologies; the employer will accept any amount of experience with the required skills.

Culture & Benefits

• Full-time employment in Bentonville, Arkansas.
• Medical, vision, and dental coverage; 401(k), stock purchase plan, and company-paid life insurance.
• Paid time off including sick leave, parental leave, family care leave, bereavement, jury duty, and voting leave.
• Short- and long-term disability, education assistance including 100% company-paid college degrees, company discounts, military service pay, and adoption expense reimbursement.
• Benefits may depend on job classification and length of employment and are subject to plan terms and change.
• Equal opportunity and drug-free workplace policies apply.