Senior Data Engineer (AI)

TL;DR

Senior Data Engineer (AI): Designing and maintaining scalable data pipelines and infrastructure for large-scale AI models with an accent on distributed systems, orchestration, and cloud-native integration. Focus on building performant data workflows, ensuring data quality, and implementing robust CI/CD processes for deep learning-driven applications.

Location: Tel Aviv

Company

hirify.global is an AI-driven company specializing in deep learning, algorithmic trading, and large-scale data solutions like the Large Market Model (LMM) for demand forecasting.

What you will do

• Design, build, and maintain high-performance data pipelines and infrastructure.
• Optimize processes for scalability and efficiency in distributed environments.
• Ensure data quality, integrity, and performance across all workflows.
• Work with cloud-native solutions and containerized environments including Docker and Kubernetes.
• Implement and manage orchestration frameworks like Airflow or Dagster.
• Develop CI/CD processes and data testing frameworks to improve pipeline reliability.

Requirements

• 5+ years of professional experience with Python and SQL.
• Proven experience building and maintaining batch/streaming ETL/ELT pipelines.
• Hands-on experience with distributed computing frameworks such as Spark, Dask, or Beam.
• Experience with major cloud platforms like AWS, GCP, or Azure.
• Proficiency with Docker, Kubernetes, and CI/CD pipelines.
• Solid understanding of software engineering practices including data structures, algorithms, and code quality.

Nice to have

• Experience with dbt, ClickHouse, or Ray.
• Familiarity with Python data libraries such as pandas, NumPy, and Apache Arrow.
• Background in functional programming.