Назад
1 день назад

Machine Learning Infrastructure Engineer, Recommendations and Search

122 574 - 259 200$
Формат работы
onsite
Тип работы
fulltime
Грейд
middle
Английский
b2
Страна
US
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Machine Learning Infrastructure Engineer, Recommendations and Search (Large-Scale ML/LLM Infrastructure): Building distributed real-time training, inference, and serving platforms for TikTok recommendations and search with an accent on multi-node parallelism, GPU efficiency, and low-latency production serving. Focus on designing resilient multi-thousand-GPU systems, co-designing generative recommendation infrastructure, and securing high-throughput multimodal data pipelines.

Location: San Jose R&D; fully in-person schedule up to 5 days a week

Salary: $122,574–$259,200 annually, with potential discretionary bonuses, incentives, and restricted stock units.

Company

TikTok USDS Joint Venture develops safety, security, data privacy, and cybersecurity capabilities for TikTok apps and U.S. user data.

What you will do

  • Implement and own large-scale distributed real-time machine learning training and inference platforms supporting TikTok recommendations and search.
  • Design and scale multi-node distributed training systems using data, tensor, and pipeline parallelism across GPU clusters.
  • Build low-latency, high-throughput model serving and optimize inference pipelines for massive live traffic.
  • Partner with applied ML researchers to develop production components for generative recommendation systems.
  • Build fault detection, asynchronous checkpointing, multimodal data processing, and storage systems for large GPU clusters.
  • Apply encryption, access controls, and tenant isolation to meet security and compliance requirements.

Requirements

  • Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related technical discipline.
  • 2+ years of professional software engineering experience with deep expertise in Python and C++ or Java, including 1+ year building and maintaining enterprise-scale machine learning infrastructure.
  • Deep familiarity with PyTorch or TensorFlow internals, GPU memory management, CUDA interactions, and InfiniBand or RoCE networking topologies.
  • Understanding of production-grade LLM training and inference tools, with hands-on profiling to eliminate I/O, compute, and network bottlenecks.
  • Strong system-level troubleshooting and debugging skills.
  • Strong communication skills, including technical design documentation and effective collaboration across team boundaries.

Nice to have

  • Experience optimizing large-scale training loops through communication-computation overlap and zero-bubble pipeline scheduling with frameworks such as Megatron or DeepSpeed.
  • Experience scaling LLM training or inference across hundreds of GPUs, including KV cache management, prefill-decoding separation, and model quantization.
  • Experience in regulated industries, sovereign cloud environments, or federal data security compliance.
  • Low-level kernel development and graph compilation experience with CUDA, Triton, Cutlass, TensorRT, or Triton Inference Server.
  • Background or interest in MLOps, Mixture of Experts routing infrastructure, and specialized hardware optimization.

Culture & Benefits

  • On-site work supports real-time decision-making, team development, and integrated execution.
  • Medical, dental, and vision insurance, 401(k) with company match, paid parental leave, disability coverage, life insurance, and wellbeing benefits.
  • 10 paid holidays, 10 paid sick days, and 17 days of paid personal time, with increasing accruals by tenure.
  • Inclusive workplace focused on curiosity, humility, resilience, continuous learning, and impact.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →