Inference Runtime Engineer (AI)

Location: On-site in San Francisco, California; remote work may be considered in the US for exceptional candidates.

Salary: $200,000–$400,000 annual salary plus equity

Company

hirify.global develops and advances vLLM as an AI inference engine, focusing on making model inference faster and more cost-efficient across models and hardware.

What you will do

• Optimize how LLM and diffusion models execute across diverse hardware and model architectures.
• Develop and improve inference runtime capabilities in vLLM.
• Implement model architectures and inference techniques from research papers.
• Contribute performant, maintainable code to complex machine-learning codebases.
• Debug inference systems and support evolving architectures such as mixture-of-experts, multimodal, and agentic models.

Requirements

• Bachelor's degree or equivalent experience in computer science, engineering, or a related field.
• Deep understanding of transformer architectures and their variants.
• Strong Python programming skills and experience with PyTorch internals.
• Experience with LLM inference systems such as vLLM, TensorRT-LLM, SGLang, or TGI.
• Ability to understand and implement model architectures and inference techniques from research papers.
• Ability to write performant, maintainable code and debug complex ML codebases.

Nice to have

• Knowledge of KV-cache memory management, prefix caching, and hybrid model serving.
• Familiarity with reinforcement-learning frameworks and algorithms for LLMs.
• Experience with multimodal inference across audio, image, video, and text.
• Contributions to open-source machine-learning or systems-infrastructure projects.
• Experience contributing core features or integrations to vLLM and related inference projects.

Culture & Benefits

• Work at the intersection of AI models and hardware on the vLLM inference engine.
• Health, dental, and vision benefits.
• 401(k) with company match.
• Equity included in the compensation package.
• Visa sponsorship is available on a case-by-case basis.