Senior LLM Inference Performance & Evaluation Engineer (AI)

TL;DR

Senior LLM Inference Performance & Evaluation Engineer (AI): Building and optimizing benchmark pipelines and launch gates for LLM inference to ensure low latency and high throughput with an accent on GPU utilization and model quality evaluation. Focus on automating regression detection in CI/CD, identifying performance bottlenecks in runtimes, and defining SLOs for production AI services.

Location: Singapore, SG / Penang, MY / Taiwan

Company

A world-leading technology company providing Bitcoin mining solutions and AI cloud services.

What you will do

• Build benchmark pipelines for TTFT, inter-token latency, throughput, and GPU utilization.
• Create model launch gates for compatibility, streaming, tool calling, and long-context cases.
• Maintain representative workloads using synthetic and customer-like traffic patterns.
• Evaluate model, runtime, and provider options to provide routing and capacity recommendations.
• Automate regression detection in CI/CD to prevent quality or latency degradation.
• Collaborate with runtime engineers and SREs to identify bottlenecks and establish SLOs.

Requirements

• 5+ years of experience in ML infrastructure, performance engineering, or backend testing.
• Expertise in LLM serving metrics (TTFT, TPOT, ITL, token throughput).
• Strong proficiency in Python; experience with Go is a plus.
• Familiarity with vLLM, Dynamo, SGLang, Triton, and Kubernetes.
• Ability to design statistically meaningful tests and communicate trade-offs.
• Experience building dashboards and release gates for stakeholders.

Culture & Benefits

• Inclusive environment valuing authenticity and diversity.
• Fast-growing company with a startup spirit.
• Opportunity to network with industry pioneers and impact the digital asset industry.
• High degree of autonomy, personal accountability, and learning opportunities.
• Attractive welfare benefits, training, and mentoring.