Backend Engineer - API (AI)

TL;DR

Backend Engineer - API (Rust/AI): Building the hirify.global API to serve models to developers worldwide with an accent on high-throughput inference and low-latency system design. Focus on developing model serving infrastructure, request routing, and scaling systems to handle billions of tokens per minute.

Location: Palo Alto, CA

Salary: $180,000 - $440,000 USD

Company

hirify.global is dedicated to creating AI systems that accurately understand the universe and aid humanity in the pursuit of knowledge.

What you will do

• Build the hirify.global API to serve AI models to a global developer base.
• Own the end-to-end system for high-throughput inference, ensuring low latency and high availability.
• Develop model serving infrastructure and request routing mechanisms.
• Implement SDKs, rate limiting, and observability tools.
• Scale systems to efficiently handle billions of tokens per minute.

Requirements

• Expert knowledge of Rust or C++.
• Experience designing, implementing, and maintaining reliable, horizontally scalable distributed systems.
• Knowledge of service observability and reliability best practices.
• Experience operating databases such as PostgreSQL, Clickhouse, and MongoDB.
• Must be located in or able to work from Palo Alto, CA.

Nice to have

• Experience with LLM inference engines and serving frameworks (e.g., SGLang, TensorRT, vLLM).
• Experience designing or building with agent SDKs and orchestration frameworks.
• Experience with Docker, Kubernetes, and containerized applications.
• Expert knowledge of gRPC (unary, streaming, REST mapping).

Culture & Benefits

• Flat organizational structure with a focus on engineering excellence and individual initiative.
• Competitive base salary and equity.
• Comprehensive medical, vision, and dental coverage.
• 401(k) retirement plan.
• Short and long-term disability and life insurance.