Principal Software Engineer (Inference)

Location: Hybrid, averaging 2 days per week from an HPE office in Spring, Texas; Durham, North Carolina; or Fort Collins, Colorado. Other HPE sites in the United States may also be considered; remote options may be considered.

Salary: $160,000–$303,000 annually in Colorado; $152,000–$349,000 annually in North Carolina and Texas. The ranges represent base salary; variable incentives may also be offered.

Company

hirify.global develops edge-to-cloud infrastructure and enterprise AI solutions, including AI Essentials for operating large language models on customer-owned infrastructure.

What you will do

• Define the technical direction and architecture of the LLM serving runtime, including engine integration, continuous batching, KV cache management and reuse, quantized execution, and distributed inference.
• Partner with performance engineering teams to improve time-to-first-token, inter-token latency, throughput per GPU, and P95/P99 tail latency.
• Design distributed serving strategies using disaggregated prefill/decode, tensor and pipeline parallelism, and KV cache offload across GPU memory, host memory, and RDMA-attached storage.
• Evaluate inference runtimes, quantization schemes, speculative decoding, and mixture-of-experts serving for adoption, in-house development, or rejection.
• Define Kubernetes orchestration for model admission, GPU scheduling and partitioning, cache-aware routing, and autoscaling.
• Mentor engineers, lead design and architecture reviews, and present technical direction to business-unit and executive audiences.

Requirements

• Production experience with LLM inference engines such as vLLM, SGLang, TensorRT-LLM, TGI, or NVIDIA NIM, including modifying engine internals.
• Comprehensive knowledge of continuous batching, paged attention, KV cache reuse and prefix caching, chunked prefill, quantization, and speculative decoding.
• Experience with tensor and pipeline parallelism, NCCL collective operations, GPU memory hierarchies, and GPU interconnect characteristics.
• Expert proficiency in Kubernetes platform architecture, including operators, custom resources, controllers, and scheduling.
• Strong Go and Python programming skills, with the ability to read, debug, and profile C++/CUDA using tools such as Nsight.
• Experience debugging and profiling multi-tier workloads such as RAG and agent applications.
• Excellent analytical, debugging, and problem-solving abilities.
• At least 12 years of software engineering experience, including more than 1 year working directly on LLM inference runtimes or production model serving.
• Degree in Computer Science or a related field.

Nice to have

• Upstream contributions to vLLM, SGLang, TensorRT-LLM, LLM-D, LMCache, or KServe.
• Experience with disaggregated prefill/decode serving or KV cache offload and reuse at scale.
• Experience with RDMA, GPUDirect Storage, InfiniBand, or RoCE.
• Experience with MIG, fractional GPU allocation, and multi-tenant GPU isolation.
• Experience delivering on-premises, air-gapped, or regulated enterprise software.

Culture & Benefits

• Health, financial, and emotional wellbeing benefits are available to employees and their families.
• Professional development programs support career growth and movement across divisions.
• HPE promotes an inclusive workplace and provides workplace accommodations for applicants with disabilities.
• The role is a full-time staff position with US employee benefits.