Senior Software Engineer (ML Inference & Numerics)

Location: Munich, Germany

Company

hirify.global is a well-funded AI systems startup developing custom silicon, hardware, and software for high-performance, low-power multimodal generative AI inference at scale for hyperscaler and neocloud data center customers.

What you will do

• Build agentic workflows that inspect models, modify implementations and numerical settings, run emulation and hardware experiments, diagnose failures, and evaluate results.
• Bring new model architectures from reference code to validated, performant execution on the company’s accelerator, converting findings into reusable model support and SDK improvements.
• Optimize precision, accumulation, quantization, kernels, KV caches, memory use, and distributed execution against latency, throughput, and model quality.
• Build conversion and emulation software, compare activations and outputs with reference models, trace accuracy regressions, and validate end-to-end evaluations.
• Own software design, implementation, testing, and delivery, including clear APIs, reproducible debugging, and durable fixes.
• Collaborate with hardware architects to evaluate numerical formats and arithmetic choices and influence future hardware and algorithm designs.

Requirements

• Deep end-to-end understanding of LLM inference, including prefill, decode, attention, KV caches, MoE, batching, parallelism, and their compute, memory, and communication implications on accelerators.
• Experience owning substantial software systems and delivering complex features from design through production.
• Strong Python skills and rigorous approaches to interfaces, abstractions, and testing.
• Ability to build and use agentic tooling, understand how agents operate effectively in large software organizations, and apply judgment about when human intervention is needed.
• Strong ownership, hypothesis-driven problem solving, cross-boundary collaboration, technical communication through specifications, design documents, and reviews, and ability to explain complex trade-offs.

Nice to have

• Expertise in debugging and optimizing complex model inference runtimes and reasoning about accuracy-performance trade-offs.
• Experience with ML compilers, accelerator SDKs, or model compression libraries, especially for non-NVIDIA accelerators such as TPU, Trainium/Inferentia, Tenstorrent, Groq, or Cerebras.
• Experience with model quantization, sparsity, codebooks, low-rank factorization, or other compression techniques.
• Contributions to distributed inference or serving frameworks such as vLLM or SGLang.

Culture & Benefits

• Work directly with hardware designers and see software decisions tested on real silicon.
• Comprehensive benefits, competitive compensation, flexible spending options, and recognition programs for employees and their families.
• Values include putting people first, ethics and integrity, ambitious goals, excellence, ownership, collaboration, and inclusion.
• hirify.global is an equal opportunity employer and considers qualified applicants without regard to protected characteristics.
• Recruitment agencies should not contact employees or leaders; unsolicited agency resumes are not accepted.