Senior Test Automation Data Engineer (Python)

TL;DR

Senior Test Automation Data Engineer (Python/Databricks): Building and maintaining automated data validation pipelines to ensure consistency and accuracy across cloud migration environments with an accent on data integrity across processing layers and storage systems. Focus on designing parity checks, validating message integrity in Kafka, and integrating quality gates into Jenkins CI pipelines.

Location: Poland (Remote/Office)

Company

hirify.global is a leading software development company providing comprehensive digital transformation services.

What you will do

• Design and implement automated data validation pipelines comparing AWS and GCP outputs using Python and Databricks.
• Develop data parity checks across ETL processing layers and validate message integrity across Kafka clusters.
• Execute object parity and stored-procedure validation for migrated data stores including S3, GCS, and Cloud SQL.
• Verify DAG execution, scheduling, retry logic, and alerting parity following Cloud Composer and Airflow migration.
• Create automated data quality gates integrated into Jenkins CI pipelines.
• Produce data validation evidence packs supporting migration acceptance and sign-off.

Requirements

• 4+ years of experience in data quality engineering and automated data validation for distributed systems.
• Proficiency in Python and advanced SQL for complex validation and reconciliation queries.
• Strong experience with Databricks, Kafka, and Redis validation.
• Expertise in object parity validation across S3 and GCS.
• Experience with CI/CD processes and Jenkins-based test orchestration.
• Strong communication skills and ability to collaborate with English-speaking client teams.

Nice to have

• Experience leveraging AI-assisted workflows and AI coding agents for validation, automation, and data analysis.

Culture & Benefits

• Equal Opportunity Employer ensuring diversity and inclusion.
• Engagement in complex, high-impact cloud migration projects.
• Flexible work arrangement with Remote/Office options.