Senior Data Engineer (AI)

TL;DR

Senior Data Engineer (AI): Designing and maintaining scalable data pipelines and infrastructure for large-scale AI models with an accent on distributed systems, orchestration frameworks, and cloud-native solutions. Focus on optimizing data flow efficiency, ensuring data quality across complex workflows, and implementing robust CI/CD and monitoring processes.

Location: Must be based in or able to commute to Wrocław, Poland (Hybrid)

Company

hirify.global is an AI-driven company leveraging deep learning and Large Market Models (LMM) to deliver algorithmic trading and real-time demand forecasting solutions.

What you will do

• Design and maintain scalable ETL/ELT pipelines for batch and streaming data.
• Optimize system performance and data integrity within distributed computing environments.
• Implement and manage orchestration frameworks to ensure reliable data flow.
• Develop CI/CD processes and automated data testing frameworks.
• Collaborate with cross-functional research and business teams to support AI development.
• Monitor and troubleshoot pipeline performance in cloud-native environments.

Requirements

• 5+ years of professional experience with Python and SQL.
• Proven experience building and maintaining complex ETL/ELT pipelines.
• Hands-on experience with distributed computing frameworks such as Spark, Dask, or Beam.
• Proficiency with containerization (Docker/Kubernetes) and cloud platforms (AWS, GCP, or Azure).
• Solid understanding of software engineering practices, TDD, and code quality.
• Must be able to work from the office in Wrocław, Poland.

Nice to have

• Experience with dbt, ClickHouse, or Ray.
• Familiarity with Python data libraries like pandas, NumPy, or Apache Arrow.
• Background in functional programming.

Culture & Benefits

• Work on cutting-edge AI and Large Market Model technology.
• Opportunity to influence data architecture in a growing, innovative company.
• Collaborative environment with cross-functional research teams.