Senior Backend Engineer (AI Inference Platform)

Location: Remote anywhere in the United States; preference for candidates in the San Francisco Bay Area or New York, where company offices are located.

Salary: $200,000–$250,000 USD per year

Company

hirify.global builds AI search infrastructure, retrieval systems, and knowledge-layer products for AI agents, applications, and enterprise workflows.

What you will do

• Own and scale the inference gateway through which product and customer model calls are routed.
• Build provider and model selection, fallback, caps, and autonomous model-routing capabilities.
• Optimize routing using prefix-cache behavior, batching, time to first token, cost, and availability.
• Implement accurate cost attribution and usage metering, including cached-token accounting.
• Build request-path enforcement that fails gracefully without interrupting customer inference calls.
• Ship inference-time changes in shadow mode and measure their production impact.

Requirements

• Production experience with model-serving stacks such as vLLM, SGLang, or similar platforms.
• Working knowledge of continuous batching, KV and prefix caching, time to first token, throughput, and p99 performance.
• Experience tuning serving infrastructure against live traffic.
• Fluency with provider APIs, tokenization, prompt-caching semantics, usage reporting, and pricing differences.
• Experience operating a proxy, gateway, or data plane on the critical path of production traffic, including streaming responses and mid-stream failures.
• Strong backend skills with Python and Kubernetes, with the ability to learn additional technologies as needed.

Nice to have

• Experience at an inference provider or GPU cloud, or contributions to an open-source serving stack.
• Experience fine-tuning and evaluating models in production.
• Experience with usage metering, billing, or cost optimization at scale.

Culture & Benefits

• San Francisco and New York hubs offer in-person gatherings and co-working sessions.
• Flexible PTO, U.S. holidays, and an end-of-year shutdown in December.
• Health insurance covering 100% of the policyholder and 75% of dependents.
• 12 weeks of paid parental leave in the U.S., a 401(k) program with a 3% immediate vesting match, and wellness benefits.
• Work-from-home and technology stipends, including support for cell phone and internet expenses.
• Opportunity to collaborate with a team working at the forefront of AI research.

Hiring process

• Compensation and leveling are determined through multiple factors assessed during the interview process.
• AI-powered tools may support scheduling, note-taking, transcription, or interview summarization; candidates may opt out where reasonably practicable.
• Employment authorization is verified through E-Verify.