Senior Test Automation Data Engineer (Python)

TL;DR

Senior Test Automation Data Engineer (Python/Databricks): Building and maintaining automated data validation pipelines for cloud migration from AWS to GCP with an accent on data integrity across processing layers and storage systems. Focus on designing data parity checks, validating Kafka message integrity, and integrating quality gates into CI/CD pipelines.

Location: Remote or Office in Bulgaria, Poland, or Romania

Company

hirify.global is a leading provider of software development and digital transformation services for global clients.

What you will do

• Design and implement automated data validation pipelines using Python and Databricks to compare AWS and GCP outputs.
• Develop data parity checks across ETL processing layers and validate message integrity across Kafka clusters.
• Verify DAG execution, scheduling, and alerting parity for Cloud Composer and Airflow migrations.
• Create automated data quality gates integrated into Jenkins CI pipelines.
• Collaborate with engineering teams on migration-wave acceptance criteria and validation scope.
• Adopt AI-assisted and agentic workflows for validation, test authoring, and defect analysis.

Requirements

• 4+ years of experience in data quality engineering and automated data validation for distributed systems.
• Proficiency in Python and advanced SQL for complex reconciliation queries.
• Strong experience with Databricks, Kafka, and Redis validation.
• Expertise in object parity validation across S3 and GCS.
• Experience with CI/CD processes and Jenkins-based test orchestration.
• English proficiency for collaboration with client-side engineering teams.
• Must be located in Bulgaria, Poland, or Romania.

Culture & Benefits

• Opportunity to work on complex cloud migration projects across diverse tech stacks.
• Equal Opportunity Employer commitment regardless of race, religion, age, or gender identity.
• Remote-friendly work options (indicated by #LI-Remote).