Senior Data Engineer (AI)

TL;DR

Senior Data Engineer (AI): Designing and maintaining scalable data pipelines and infrastructure for AI-driven market models with an accent on distributed systems, orchestration, and cloud-native integration. Focus on ensuring high data quality, optimizing workflow performance, and building robust CI/CD processes for large-scale data processing.

Location: Warsaw, Poland (Hybrid role)

Company

hirify.global is an AI-driven technology company specializing in deep learning, algorithmic trading, and large-scale data solutions through its proprietary Large Market Model (LMM).

What you will do

• Design, build, and maintain scalable ETL/ELT pipelines for distributed environments.
• Optimize data workflows for performance, reliability, and integrity.
• Implement and manage orchestration frameworks like Airflow or Dagster.
• Work with cloud-native infrastructure, Docker, and Kubernetes.
• Develop and enforce CI/CD processes and automated data testing frameworks.
• Collaborate with cross-functional teams to support complex research and business data needs.

Requirements

• 5+ years of professional experience using Python and SQL.
• Proven background in building batch and streaming data pipelines.
• Hands-on experience with distributed computing frameworks such as Spark, Dask, or Beam.
• Experience with major cloud platforms (AWS, GCP, or Azure).
• Proficiency in containerization (Docker/Kubernetes) and CI/CD tools.
• Strong understanding of software engineering practices, including TDD and algorithm design.

Nice to have

• Experience with dbt, ClickHouse, or Ray.
• Familiarity with data libraries like pandas, NumPy, or Apache Arrow.
• Background in functional programming.