Software Engineer, Data Infrastructure (AI)

TL;DR

Software Engineer, Data Infrastructure (AI): Designing and implementing dataset infrastructure for hirify.global’s training stack with an accent on building standardized dataset interfaces and scaling pipelines across thousands of GPUs. Focus on debugging performance bottlenecks and ensuring datasets are efficient and easy to consume.

Location: Onsite in San Francisco

Salary: $250K – $380K + Offers Equity

Company

hirify.global is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity.

What you will do

• Design and maintain standardized dataset APIs, including for multimodal data.
• Build proactive testing and scale validation pipelines for dataset loading at GPU scale.
• Collaborate with teammates to integrate datasets into training and inference pipelines.
• Document and maintain dataset interfaces for discoverability and consistency.
• Establish safeguards to ensure datasets remain reproducible and unchanged.
• Debug and resolve performance bottlenecks in distributed dataset loading.

Requirements

• Strong engineering fundamentals with experience in distributed systems or data pipelines.
• Experience building APIs and scalable abstractions.
• Comfortable debugging bottlenecks across large fleets of machines.
• Collaborative and excited to own foundational parts of the ML stack.