Software Engineer (AI Infrastructure)

TL;DR

Software Engineer (AI Infrastructure): Designing and building core backend systems for a generative AI platform with an accent on scalable inference engines, distributed training, and data pipelines. Focus on ensuring reliability, performance, and availability of LLM services while bridging the gap between proprietary inference engines and customer needs.

Location: Must be based in New York, NY or San Mateo, CA

Company

hirify.global is a high-growth Series C startup building cutting-edge generative AI infrastructure, backed by top investors and founded by veterans of Meta PyTorch and Google Vertex AI.

What you will do

• Contribute to the design and development of scalable backend infrastructure for distributed training and model serving.
• Build and maintain core services including LLM CI/CD pipelines, control planes, and model serving systems.
• Optimize performance, cost efficiency, and reliability across compute, storage, and network layers.
• Develop frameworks and safeguards to ensure industry-leading model quality.
• Collaborate with performance, training, and product teams to translate research requirements into infrastructure solutions.
• Participate in technical reviews and continuous integration/deployment processes.

Requirements

• Bachelor’s degree in Computer Science, Engineering, or equivalent practical experience.
• 3+ years of software engineering experience focusing on infrastructure or ML systems.
• Strong proficiency in Python or Go.
• Proven experience with ML infrastructure and tooling such as PyTorch, MLflow, Vertex AI, SageMaker, or Kubernetes.
• Foundational understanding of LLM technologies including context length, prefill, and KV cache management.

Nice to have

• 5+ years of experience in infrastructure or ML systems engineering.
• Hands-on experience with open-source inference engines like vLLM, Sglang, or TRT-LLM.
• Experience building large-scale MLOps infrastructure.
• Track record of contributions to open-source ML projects.

Culture & Benefits

• Work at the forefront of AI infrastructure with bleeding-edge technology.
• Join an ambitious, collaborative team with a flat structure and high ownership.
• Direct impact on how businesses and developers globally harness generative AI.
• Opportunity to learn from world-class engineers and AI researchers.