Senior AI Research Engineer (LLM Training & Inference Infrastructure)

Location: Paris, France; hybrid

Company

Datadog provides an observability and security platform that unifies visibility across applications, infrastructure, data, models, and security, including AI-powered solutions.

What you will do

• Design, build, and operate distributed infrastructure for large-scale LLM training and inference, including Mixture-of-Experts models.
• Scale training through data, tensor, pipeline, and expert parallelism, memory optimization, checkpointing, and failure recovery.
• Build inference, rollout, evaluation, and reinforcement-learning systems, including execution environments and sandboxes for agent post-training.
• Benchmark rollout speed, training throughput, and learning behavior; maintain dashboards that expose performance regressions.
• Improve job submission, scheduling, quotas, GPU utilization, and observability for researchers.
• Collaborate with platform teams on GPU cluster capacity, networking, scheduling, and distributed-systems reliability.

Requirements

• Deep experience with distributed systems and ML infrastructure for training and inference at scale.
• Proficiency in Python and at least one systems language such as Rust, C++, or Go.
• Experience operating large-scale training or serving infrastructure, or schedulers and orchestration systems for GPU or other high-performance workloads.
• Understanding of distributed inference throughput and latency constraints, including communication, memory, scheduling, and stragglers.
• Strong debugging, profiling, and incident-diagnosis skills; on-call experience is beneficial.
• Ability to explain design and performance trade-offs to technical and non-technical audiences.
• Prior machine-learning model training experience is valuable but not essential for candidates with strong distributed-systems or HPC backgrounds.

Nice to have

• Experience with Ray, Kubernetes, Slurm, Megatron-LM, TorchTitan, or DeepSpeed.
• Experience with LLM inference engines such as vLLM or SGLang, including batching and KV-cache management.
• Experience with reinforcement-learning training systems and rollout infrastructure.
• Experience with NCCL, RDMA, low-latency networking, GPU benchmarking, profiling, and cluster performance monitoring.
• Experience building sandboxed code execution or large-scale evaluation infrastructure.
• Contributions to open-source ML or distributed-systems projects.

Culture & Benefits

• Competitive global benefits, new-hire RSUs, and an employee stock purchase plan.
• Close collaboration across Datadog offices in New York City and Paris.
• Opportunities to attend and present at conferences and meetups.
• Intra-departmental mentor and buddy program, inclusive culture, and employee resource groups.
• Benefits vary by country of employment and the nature of employment.