Member of Technical Staff (Data Infrastructure)

TL;DR

Member of Technical Staff (Data Infrastructure): Building and operating massive-scale data systems for training generative AI models with an accent on peta-to-exabyte scale data movement. Focus on designing scalable data loaders, optimizing petabyte-scale storage, and resolving performance bottlenecks across thousands of GPUs.

Location: Must be based in or able to work from Freiburg, Germany, or San Francisco, USA

Salary: $180,000–$300,000 USD

Company

hirify.global builds frontier generative image and video models used by millions worldwide.

What you will do

• Develop and maintain scalable data loaders for training models on thousands of GPUs.
• Engineer efficient storage and retrieval systems for petabyte-scale datasets.
• Implement multi-cloud object storage abstractions and manage large-scale data migrations.
• Debug and resolve performance bottlenecks within distributed data loading pipelines.
• Work with video and image data processing at massive scale.

Requirements

• Proven experience operating data pipelines at petabyte scale.
• Deep understanding of PyTorch DataLoader internals and data loading optimization.
• Experience processing datasets containing millions of files.
• Ability to debug distributed system bottlenecks across large fleets of machines.
• Extensive experience with object storage such as S3, Azure Blob, or GCS.

Nice to have

• Experience with streaming dataset formats like WebDataset.
• Knowledge of video codec internals and frame-accurate seeking.
• Practical experience with Slurm and Kubernetes for job orchestration.
• Expertise in performance tuning object storage across different cloud providers.

Culture & Benefits

• Work at the frontier of visual AI on models used by millions.
• Focus on research rigor and open science.
• Opportunity to solve complex data challenges at a global scale.
• Equity package included with compensation.