Software Engineer - Model API's (AI)

TL;DR

Software Engineer - Model API's (AI): Designing and operating the Model APIs infrastructure for hosted API endpoints of open-source models with an accent on advanced inference capabilities and performance optimization. Focus on profiling, optimizing, and productionizing performance improvements across distributed systems.

Location: San Francisco, New York

Compensation Range: $150K - $230K

Company

hirify.global powers inference for AI companies by providing flexible infrastructure and developer tooling.

What you will do

• Design, build, and operate the Model APIs surface.
• Profile and optimize TensorRT-LLM kernels and analyze CUDA kernel performance.
• Productionize performance improvements across runtimes.
• Build comprehensive benchmarking frameworks.
• Implement platform fundamentals like API versioning and authentication.
• Collaborate closely with other teams for robust model serving experiences.

Requirements

• 3+ years experience with distributed systems or large-scale APIs.
• Proven track record of low-latency, reliable backend services.
• Strong debugging skills for complex systems.
• Excellent written communication skills.

Nice to have

• Experience with LLM runtimes or open-source inference engines.
• Knowledge of Kubernetes or service meshes.
• Background in developer-facing infrastructure.

Culture & Benefits

• Competitive compensation with equity options.
• 100% coverage of medical, dental, and vision insurance.
• Generous PTO policy including a Winter Break.
• Paid parental leave.
• Company-facilitated 401(k).