Senior ML Engineer (Inference Stack Optimization)

Location: Remote-first role available in Austria, France, Germany, Italy, the Netherlands, Poland, Spain, or the United Kingdom

Company

hirify.global builds cloud infrastructure and Kimchi inference systems for serving and optimizing large language models on shared GPU fleets and customer Kubernetes clusters.

What you will do

• Optimize throughput through continuous batching, chunked prefill, speculative decoding, CUDA-graph capture, and scheduler tuning across GPU SKUs.
• Reduce time-to-first-token and inter-token latency by profiling compute, memory bandwidth, scheduling, storage, and network bottlenecks.
• Improve KV-cache utilization with hierarchical and paged attention, prefix caching, eviction policies, and quantized KV caches.
• Accelerate cold starts for terabyte-scale checkpoints using O_DIRECT loaders, fastsafetensors-class tooling, JIT caches, and shared-filesystem read paths.
• Scale distributed inference across nodes through network-aware placement, prefill/decode disaggregation, and storage-efficient checkpointing strategies.
• Set technical direction through benchmarks, reproducible experiments, technical writeups, and decisions about what to adopt or build.

Requirements

• 5+ years building production ML systems, with depth in inference or training infrastructure.
• Production-grade Python experience building services.
• Hands-on experience with vLLM, SGLang, or TensorRT-LLM and a strong understanding of engine performance on different GPUs.
• Experience measuring quantization regressions and working with INT8/INT4 or FP8/FP4 across weights, activations, and KV caches.
• Experience with multi-GPU and multi-node inference, tensor and expert parallelism, collective communication, and distributed failure modes.
• Knowledge of page caches, O_DIRECT, network filesystems, instrumentation, and performance benchmarking at terabyte scale.

Nice to have

• Experience with LLM routing, RouterBench-style evaluations, and cost, quality, and latency Pareto analysis.
• Production experience with calibration, abstention, conformal methods, or online A/B loops with model drift monitoring.

Culture & Benefits

• Flexible, remote-first environment with a globally distributed team.
• Competitive salary, equity options, equipment budget, and extra days off.
• 10% of work time allocated to personal projects or self-improvement.
• Learning budget covering professional development, conferences, and courses.
• Annual hackathon, team-building budget, and company events.

Hiring process

• Recruiter screening call, hiring manager interview, and technical system-design interview.
• Live coding and culture-check interview with an executive.
• A background check may be conducted at the final recruitment stage. Visa sponsorship and work-permit sponsorship are not provided.