AI Researcher - Pre-training

Location: Heidelberg, Germany; hybrid workplace

Company

hirify.global develops European foundation models for finance, manufacturing, and public administration, with a focus on German-language understanding, regulatory requirements, and reliable high-stakes use.

What you will do

• Design and run ablations for optimizers, schedules, initialization, and other training-recipe components.
• Implement and evaluate PyTorch-based architecture changes for convergence, stability, and training efficiency.
• Run proxy experiments and contribute to scaling laws, hyperparameter transfer, and scale-up methods.
• Monitor large-scale training runs and debug loss spikes, divergence, and distributed training failures.
• Reproduce promising research results and collaborate with Compute Performance, Data, Evaluation, and Post-Training teams.

Requirements

• Proficiency in Python and hands-on experience with PyTorch.
• Strong machine-learning fundamentals demonstrated through research, a PhD, a strong master's thesis, publications, open-source contributions, or substantial professional or personal projects.
• Solid mathematical background in linear algebra, probability, and optimization.
• Understanding of transformer architectures and hands-on experience pre-training LLMs from scratch, including at small scale.
• Ability to design careful experiments, report noisy and negative results honestly, and explain evidence clearly.
• Ability to learn unfamiliar technical areas quickly, write clean maintainable code, and create reproducible experiments.
• Clear communication, early escalation of problems, and active use of feedback.

Nice to have

• Experience with multi-GPU or multi-node training, distributed frameworks such as Megatron-LM, DeepSpeed, or torchtitan, and data, tensor, pipeline, or expert parallelism.
• Familiarity with scaling laws, hyperparameter transfer such as µP, optimizers, or training stability.
• Exposure to Mixture-of-Experts, efficient attention variants, or other inference-efficient architectures.
• Publications at NeurIPS, ICML, or ICLR, or contributions to open-source machine-learning training libraries.
• Strong curiosity about hardware and systems constraints affecting large-scale training; low-level kernel optimization is not required.

Culture & Benefits

• Hybrid work in Heidelberg, Germany.
• Ownership, autonomy, empowerment, flat structure, and supportive management.
• Close collaboration on complex technical problems through pairing and mob programming.
• Hands-on work with large-scale clusters containing thousands of NVIDIA Blackwell GPUs.
• Progressive onboarding through pairing with experienced researchers and increasing ownership.