Senior Performance Engineer (AI)

Location: Hybrid in Heidelberg, Germany, with regular on-site collaboration required

Company

hirify.global develops foundation models for finance, manufacturing, and public administration, with a focus on German language understanding, European regulatory requirements, and reliable performance in high-stakes settings.

What you will do

• Profile training loops with PyTorch Profiler, Nsight Systems, and Nsight Compute to identify system- and kernel-level bottlenecks.
• Maximize hardware utilization and training throughput across large-scale GPU clusters with thousands of NVIDIA Blackwell GPUs.
• Configure and tune tensor, data, hierarchical, fully sharded, and expert parallelism strategies for large-scale LLM training.
• Optimize load balancing and communication-to-computation trade-offs while minimizing critical-path bottlenecks.
• Collaborate with AI researchers on model architectures designed for hardware efficiency without compromising convergence.

Requirements

• Proficiency in Python and PyTorch.
• Strong engineering background in parallel and distributed systems.
• Hands-on experience with modern machine learning techniques, especially large language models and their life cycle.
• Deep understanding of the CUDA programming model and experience with distributed programming APIs such as NCCL or MPI.
• Experience analyzing profiling traces with PyTorch Profiler and NVIDIA Nsight.
• Availability for regular on-site collaboration in Heidelberg, Germany.

Nice to have

• Contributions to distributed training frameworks such as TorchTitan, Megatron-LM, or DeepSpeed.
• Familiarity with low-precision formats including MXFP4 and MXFP8 and their impact on numerical stability and throughput.
• Experience with NCCL communication primitives, NVSHMEM, or CUDA IPC performance.
• Experience implementing and optimizing transformer-based model training.
• Experience with the NVIDIA Blackwell architecture.

Culture & Benefits

• Ownership, autonomy, empowerment, and a flat organizational structure.
• Hybrid working model with flexible working hours.
• 30 days of paid vacation.
• Fitness and wellness offerings through Wellhub and mental health support through nilo.health.
• Subsidized company pension plan, Germany-wide transportation ticket, technical equipment budget, and JobRad bike lease.