Machine Learning System Engineer (AI)

Location: Remote in the United States, with listed locations in Seattle and San Francisco.

Base salary: $147,906–$193,100 for Zone C, $160,380–$209,385 for Zone B, or $178,200–$232,650 for Zone A.

Company

hirify.global develops software products that help teams collaborate and manage different types of work.

What you will do

• Design and implement scalable distributed infrastructure for model serving, including load balancing, auto-scaling, batch scheduling, and global KV caching.
• Optimize model inference latency and throughput under production workloads.
• Build reliable, high-concurrency serving systems handling billions of requests.
• Benchmark, fine-tune, and accelerate inference engines using low-level and algorithmic optimizations.
• Create CI/CD infrastructure for model deployment and inference engine updates.
• Partner with senior ML engineers to fine-tune and deploy open-source LLMs.

Requirements

• 3+ years of software engineering experience.
• Deep low-level systems programming experience with C, C++, or Rust.
• Experience with large-scale, high-concurrency production serving.
• Experience with GPU inference engines such as vLLM, SGLang, Triton, or TensorRT-LLM.

Nice to have

• 1+ years of system performance optimization experience.
• Experience with GPU kernels, quantization, speculative decoding, or distillation.
• Experience testing, benchmarking, and improving inference service reliability.
• Experience designing CI/CD infrastructure for inference.
• Strong background in batching, caching, load balancing, and parallelism.

Culture & Benefits

• Choice of working remotely or from an office in the United States.
• Health and wellbeing resources.
• Paid volunteer days.
• Potential eligibility for benefits, bonuses, commissions, and equity.
• Accommodations and adjustments are available during the recruitment process.

Hiring process

• Identity verification may be required as a condition of employment.