Staff Machine Learning Engineer (AI Serving)

TL;DR

Staff Machine Learning Engineer (AI Serving): Designing and implementing a large-scale, low-latency GPU-based model serving system for search, ranking, and LLMs with an accent on scalability and high throughput. Focus on building a high-performance feature hydration system, GPU model export frameworks, and optimizing LLM serving at scale.

Location: Remote (United States)

Salary: $253,300 - $354,600 USD

Company

hirify.global is a community of communities and one of the internet's largest sources of information, hosting over 100,000 active communities.

What you will do

• Lead the end-to-end design and maintenance of a GPU-based model serving system supporting millions of QPS.
• Develop Generative AI systems in cloud-based production environments using Kubernetes at scale.
• Build a high-performance feature hydration and processing system including routing, caching, and batching.
• Create a unified GPU model export framework to optimize trained models for inference.
• Implement real-time ML observability to track feature and model performance.
• Develop an E2E inference performance benchmarking framework.

Requirements

• 7+ years of experience in ML Engineering, AI Platform Engineering, or Cloud AI Deployment.
• Deep experience operating Kubernetes at scale.
• Proficiency in Python and Go.
• Strong experience with cloud technologies including AWS, Google Cloud Storage, and Terraform.
• Expertise with modern AI/ML frameworks such as Pytorch, Triton, Dynamo, or vLLM.
• Must be based in the United States.

Culture & Benefits

• Comprehensive healthcare benefits and income replacement programs.
• 401k with employer match.
• Flexible vacation and paid volunteer time off.
• Generous paid parental leave.
• Mental health, coaching, and family planning support.
• Global benefit programs supporting professional development and caregiving.