Senior AI Researcher in Pre-training (f/m/d)

Location: Hybrid, Heidelberg, Germany; regular on-site collaboration required

Company

hirify.global develops foundation models for finance, manufacturing, and public administration, with a focus on German language understanding, European regulatory requirements, and reliable use in high-stakes settings.

What you will do

• Own core elements of foundation-model training recipes, including optimizers, schedules, initialization, and PyTorch-based architecture improvements.
• Develop scaling laws and proxy-experiment methodologies to predict the behavior of multi-thousand-GPU training runs.
• Investigate convergence issues, loss spikes, divergence, numerical instability, and distributed-system failures.
• Implement efficient and reliable model architectures and maintain production-quality research code.
• Collaborate with Compute Performance, Data, Evaluation, and Post-Training teams on system-model co-design.

Requirements

• Strong proficiency in Python and deep familiarity with PyTorch-based training workflows.
• Strong machine learning research and software engineering track record through shipped models, open-source contributions, or published research.
• Strong mathematical foundation with the ability to reason about optimization, scaling behavior, and training dynamics.
• Deep understanding of transformer training dynamics and large distributed training jobs.
• Hands-on experience pre-training large models, such as 7B+ parameter models, on substantial infrastructure, such as 100+ GPU clusters.
• Must be able to work in Germany and collaborate regularly on site in Heidelberg.

Nice to have

• Experience with distributed training frameworks such as Megatron-LM, DeepSpeed, or torchtitan.
• Knowledge of scaling laws, hyperparameter transfer, and predictive scaling methods.
• Experience profiling distributed jobs and diagnosing optimizer pathologies or training anomalies.
• Exposure to sparse training approaches such as Mixture-of-Experts.
• Top-tier publications, impactful open-source contributions, or significant shipped technical work.

Culture & Benefits

• Ownership, autonomy, and empowerment within a flat organizational structure.
• Close collaboration on complex technical problems through pair work and mob programming.
• 30 days of paid vacation and flexible working hours.
• Hybrid work model with wellness and mental health support.
• Subsidized company pension plan, Germany-wide transportation ticket, technical equipment budget, and JobRad bike leasing.