mcp/api
Plus
Включить Plus
Назад
Company hidden
4 дня назад

Data Engineer (AWS/PySpark)

Формат работы
hybrid
Тип работы
project
Грейд
senior
Английский
b2
Страна
US
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Data Engineer (AWS/PySpark): Design and maintain scalable ETL pipelines, real-time Kafka streaming solutions, and serverless data processing workflows with an accent on large-scale data transformation, cloud data warehousing, and infrastructure automation. Focus on optimizing EMR and Athena workloads, managing Redshift data models, implementing pipeline monitoring, and securing AWS data access with Lake Formation, IAM, and KMS.

Location: Hybrid in Charlotte, NC 28206, United States.

Contract: 12-month W-2 contract; candidates must work on W-2 only. C2C and 1099 arrangements are not accepted.

Company

hirify.global provides staffing services and is hiring for a contract data engineering role.

What you will do

  • Design, develop, and maintain scalable ETL pipelines with AWS Glue, EMR, PySpark, and Lambda.
  • Implement real-time data streaming with Confluent Kafka, including source and sink connectors.
  • Develop and optimize PySpark jobs and Athena queries for large-scale data processing and analytics.
  • Manage Redshift data warehouses, including data modeling and performance tuning.
  • Provision AWS infrastructure with Terraform and implement monitoring and alerting with CloudWatch.
  • Collaborate with analysts, data scientists, and business stakeholders while applying data governance and security practices with Lake Formation, IAM, and KMS.

Requirements

  • Advanced proficiency in Python 3.8+ for data processing, scripting, and automation.
  • Extensive experience with PySpark and EMR clusters for distributed, large-scale data processing.
  • Hands-on experience with AWS Lambda, AWS Glue, Amazon Athena, Amazon Redshift, Amazon S3, and CloudWatch.
  • Experience with Confluent Kafka, including Schema Registry, source and sink connectors, producers, and consumers.
  • Experience with Terraform for AWS infrastructure provisioning.
  • Strong understanding of distributed computing and big data processing frameworks.
  • Bachelor’s degree in Computer Science, Data Engineering, Information Technology, or a related field.
  • Experience with Agile development methodologies, strong problem-solving and analytical skills, and strong communication and collaboration abilities.
  • Experience with EMR Notebooks or Jupyter, VS Code or a similar IDE, Git, AWS CLI, boto3, DBeaver, or Data Studio.

Nice to have

  • Knowledge of AWS EventBridge and Step Functions for workflow orchestration.
  • Familiarity with EMR runtime and execution roles for secure data access.
  • Experience with DynamoDB for NoSQL data storage.
  • Knowledge of API Gateway for building data APIs.

Culture & Benefits

  • Hybrid work schedule in Charlotte, North Carolina.
  • 12-month W-2 contract arrangement.

Hiring process

  • Applicants must be able to work through hirify.global on a W-2 basis; C2C and 1099 applications are excluded.

Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →

Categories

Jobs in the USAPython jobsTerraform jobsAWS jobs