Data Engineer (GCP)

TL;DR

Data Engineer (GCP): Developing automated ETL processes and data pipelines for machine learning with an accent on data ingestion, cleaning, and preparation. Focus on building scalable data ingestion frameworks, integrating diverse data sources, and supporting Data Science teams with high-quality, transformed datasets.

Location: Colombia

Company

hirify.global is a global digital accelerator providing high-value solutions with over 20 years of experience and a multicultural innovation-focused culture.

What you will do

• Build and maintain automated ETL processes and complex data pipelines.
• Prepare, clean, and transform data for Machine Learning models and Data Science teams.
• Manage data ingestion from diverse sources including SQL databases, ERP systems, and file structures.
• Implement data quality controls and manage data annotation workflows in GCP.
• Collaborate on distributed system architectures for high-volume data processing.
• Maintain CI/CD pipelines and version control systems.

Requirements

• 3+ years of experience with SQL for ETL and complex database scripting.
• 3+ years of programming experience with Python for ETL and data pipelines.
• 2+ years of hands-on experience with GCP.
• Expertise in data extraction from at least 3 different source types such as SQL, Oracle, Snowflake, or ERP systems.
• 2+ years of experience in data cleansing, wrangling, and quality assurance.
• Proficiency in CI/CD tools, Git, and workflow orchestration like Airflow.

Nice to have

• Experience with Spark for distributed systems and high data volumes.
• Knowledge of Big Data technologies including Hadoop and NoSQL databases like MongoDB or CosmosDB.
• Familiarity with Azure cloud services and API management.

Culture & Benefits

• Professional growth opportunities within a dynamic, global environment.
• Exposure to large-scale, international digital transformation projects.
• Attractive benefits plan and statutory employment benefits.
• Culture centered on continuous learning and innovation.