ML Systems Engineer (On-Device Inference)

TL;DR

ML Systems Engineer (On-Device Inference): Building and optimizing high-performance on-device inference engines and model conversion toolkits with an accent on implementing new model architectures and optimized kernels. Focus on writing high-performance code, supporting new modalities, and advanced decoding methods.

Location: Remote (must be based in SF or Europe)

Company

Mirai builds the fastest on-device inference engine for Apple Silicon, backed by leading AI-focused funds and individuals, and advised by a former Apple Distinguished Engineer.

What you will do

• Work across the inference engine and model conversion toolkit.
• Implement new model architectures and support new modalities.
• Write optimized Metal / Vulkan kernels.
• Build a wide range of features, including function calling and batch decoding.

Requirements

• Proficiency with JAX / Equinox / Pallas stack.
• Strong Rust systems programming skills with a focus on developer experience.
• Experience writing Metal / Vulkan kernels.
• Deep understanding of Transformer / SSM / Diffusion / Vision language models.
• Expertise in advanced speculative decoding methods and trellis-based quantization approaches.
• Strong background in linear algebra, optimization methods, and probability theory.

Culture & Benefits

• Work in a small, senior, and deeply technical team of 14 people.
• Opportunity to ship fast and own problems end-to-end.
• Founded by proven entrepreneurs from successful consumer AI companies.
• Advisors include a former Apple Distinguished Engineer.