Distinguished Engineer - AI (AI)

TL;DR

Distinguished Engineer - AI (AI): Architecting and shipping next-generation AI infrastructure and inferencing platforms serving millions of requests with uncompromising latency, throughput, and reliability requirements with an accent on AI/ML inferencing at scale and distributed systems foundations. Focus on translating high-stakes business problems into elegant, defensible architectures and inventing solutions where standard approaches fail.

Location: Hybrid in San Jose, California, United States

Salary: $255,850 - $380,600 USD

Company

hirify.global is a leader in intelligent data infrastructure, delivering unified storage, integrated data services, and solutions to unlock the full potential of data for AI to multicloud.

What you will do

• Provide the technology strategy to accelerate the pace of innovation within hirify.global and the Industry.
• Define the roadmap and long-term vision, derived from key business priorities, technology, and competitive trends.
• Partner with Product management and engineering to define and deliver next-generation products.
• Influence executive management and engineering, acting as a force multiplier across engineers.
• Mentor senior and principal engineers and be the technical bar-raiser for senior and principal technical roles.
• Evangelize for hirify.global products externally, becoming an industry recognized authority on related technologies.

Requirements

• Deep hands-on experience with high-performance inference engines (TensorRT, vLLM, ONNX Runtime, Triton) and model optimization.
• Proven track record building and architecting RAG pipelines and optimizing retrieval-augmented generation workflows.
• 15+ years architecting fault-tolerant, low-latency distributed systems with expert-level understanding of consensus protocols.
• Experience building enterprise-grade or SaaS platforms specifically designed for AI/ML workloads.
• Deep familiarity with GPU/TPU cluster orchestration, memory hierarchy optimization, and heterogeneous compute scheduling.
• Expertise in RDMA, DPDK, kernel bypass techniques, and custom protocols for inter-service communication.
• Hardened multi-tenant ML infrastructure with robust isolation, end-to-end encryption, and fine-grained RBAC/ABAC.
• Expert in Kubernetes scheduling extensions, service mesh for AI microservices, and API gateway patterns.

Culture & Benefits

• Hybrid working environment designed to strengthen connection, collaboration, and culture for all employees.
• Comprehensive benefits package including Health Insurance, Life Insurance, Retirement Plans, Paid Time Off, and performance-based incentives.
• Employee stock purchase plan and/or restricted stocks (RSU’s).
• Flexibility to balance professional ambition with your personal life.
• Committed to Equal Employment Opportunity (EEO) and compliance with all federal, state, and local laws.