Data Engineer (GCP)
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
TL;DR
Data Engineer (GCP): Designing and developing an enterprise IAM Data Lake platform on Google Cloud Platform with an accent on scalable data ingestion pipelines and large-scale data processing. Focus on building PySpark applications, orchestrating complex workflows with Apache Airflow, and implementing Change Data Capture patterns.
Location: Hybrid (Irving, TX or Ohio)
Company
is a professional staffing and IT services provider specializing in technical recruitment and workforce solutions.
What you will do
- Design, build, and maintain scalable Data Lake solutions within Google Cloud Platform (GCP).
- Develop batch and streaming data ingestion pipelines using GCP-native services and big data technologies.
- Build and optimize data processing workflows, including data models and transformation processes.
- Develop PySpark-based data processing applications and orchestrate workflows via Apache Airflow.
- Implement and manage CI/CD pipelines and Pub/Sub-based streaming architectures.
- Collaborate with architecture, security, and business teams to ensure data solutions align with enterprise standards.
Requirements
- 4+ years of experience with Google Cloud Platform (GCP).
- 4+ years of experience with PySpark and distributed data processing.
- 4+ years of experience implementing CI/CD practices and deployment automation.
- 2+ years of experience building data pipelines and using Apache Airflow.
- Experience developing and integrating APIs.
- Employment eligibility: W2 only, no C2C.
Nice to have
- Experience with Hadoop Ecosystem and HDFS.
- Familiarity with Change Data Capture (CDC) methodologies and incremental ingestion.
- Experience building enterprise-scale IAM or security-related data platforms.
- Knowledge of data governance, lifecycle management, and access control within GCP.
Culture & Benefits
- 12-month contract duration.
- Opportunity to work with modern cloud-native big data technologies.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →