Data Engineer

TL;DR

Data Engineer: Designing, building, and maintaining scalable data pipelines for enterprise analytics with an accent on Databricks and Apache Spark. Focus on implementing real-time CDC solutions, optimizing ETL/ELT processes, and ensuring data integrity across distributed systems.

Location: Must be based in Colombia

Company

hirify.global is a premier partner for global enterprises, specializing in digitization, automation, and the development of next-generation digital products.

What you will do

• Design, build, and maintain scalable data pipelines using Databricks and Apache Spark.
• Develop and optimize ETL/ELT processes for diverse data source ingestion.
• Implement Change Data Capture (CDC) solutions for real-time data synchronization.
• Write clean, efficient Python code for data processing and automation.
• Collaborate with architects and analysts to translate business requirements into robust designs.
• Monitor, troubleshoot, and optimize pipeline performance and data quality.

Requirements

• Must be based in Colombia
• 3+ years of experience in data engineering.
• Strong hands-on experience with Databricks and Apache Spark.
• Proficiency in Python and SQL.
• Experience implementing CDC solutions in production environments.
• Solid understanding of data modeling and warehousing concepts.

Nice to have

• Experience with cloud platforms like AWS or Azure.
• Familiarity with orchestration tools such as Airflow.
• Knowledge of streaming technologies like Kafka.
• Experience working in Agile/Scrum environments.

Culture & Benefits

• Opportunity to work on global enterprise digital transformation projects.
• Collaborative environment working with cross-functional teams.
• Focus on data engineering best practices and professional growth.
• Commitment to equal opportunity and inclusive hiring practices.