Назад
23 дня назад

Staff Site Reliability Engineer (AI)

Формат работы
onsite
Тип работы
fulltime
Грейд
senior
Английский
b2
Страна
US
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Staff Site Reliability Engineer (AI): Building a GPU management and scheduling platform for nearly 30 models on heterogeneous hardware with an accent on metrics-driven orchestration, admission control, autoscaling, and production reliability. Focus on designing real-time control loops, operating secure cloud infrastructure, and developing Python and Go systems for efficient inference capacity management.

Location: Menlo Park, CA; on-site

Company

Hippocratic AI is building a healthcare-only, safety-focused LLM and production-grade AI platform designed to improve patient outcomes.

What you will do

  • Design and build a GPU management and scheduling platform for approximately 30 models running on heterogeneous hardware.
  • Build metrics pipelines and decision logic for GPU utilization, admission control, load shedding, and autoscaling.
  • Develop cloud orchestration systems and operators in Python and Go to manage model fleets.
  • Architect and operate scalable, fault-tolerant, secure production systems on AWS, GCP, or Azure.
  • Build infrastructure automation, Terraform and CI/CD deployment pipelines, and monitoring, logging, and alerting systems.
  • Develop healthcare AI security and compliance policies, resolve complex operational issues, and mentor engineers.

Requirements

  • 10+ years of professional experience across site reliability or DevOps engineering and software engineering.
  • Computer Science degree from a top CS program.
  • Strong software engineering fundamentals, including building orchestration and scheduling systems in Python and/or Go.
  • Experience designing metrics-driven systems and control loops for autoscaling, load shedding, or admission control.
  • Deep experience with infrastructure automation and CI/CD, including Terraform, GitLab CI/CD, or similar.
  • Hands-on production experience with at least one major cloud platform: AWS, GCP, or Azure.
  • Strong knowledge of Docker and Kubernetes.
  • Experience with monitoring and logging stacks such as ELK, Grafana, Datadog, or similar.
  • Familiarity with secrets management and security tools such as HashiCorp Vault, AWS KMS, or Azure Key Vault.
  • Excellent problem-solving, independent working, collaboration, communication, and interpersonal skills.

Nice to have

  • Experience managing GPU fleets or scheduling workloads across heterogeneous accelerators.
  • Familiarity with ML inference serving and model deployment, such as Triton, KServe, or Ray Serve.
  • Experience with Kubernetes autoscaling internals, including HPA/VPA, custom metrics, and custom controllers.
  • Experience implementing HIPAA and SOC 2 compliance.
  • Experience operating in an HPC environment.
  • Bachelor's or Master's degree in Computer Science, Computer Engineering, or a related field.

Culture & Benefits

  • Work on clinically safe, production-grade AI systems for healthcare.
  • Collaborate with physicians, hospital leaders, AI researchers, and experienced technology professionals.
  • Join a safety-focused organization building a healthcare-only LLM.
  • Equal opportunity workplace with application accommodations available during the hiring process.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →