У роли понятная зона ответственности, измеримые цели и прямое участие в развитии ИИ для здравоохранения, но быстрый темп и работа только в офисе подойдут не всем.
Только офис
Понятная ответственность
На переднем крае ИИ
Сильная поддержка
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Подключить Plus
Описание вакансии
Текст:
Hirify AI/Оригинал
TL;DR
Technical Program Manager (Frontier AI Data Labeling): Running end-to-end labeling, annotation, rater calibration, and evaluation programs that feed healthcare AI model training and release decisions with an accent on audio and conversation data, clinician-labeled datasets, preference data, and RLHF. Focus on scaling expert rater workforces, designing quality systems and rubrics, and solving throughput, cost, and annotation reliability challenges under live model-development timelines.
Location: Menlo Park, California, United States; onsite
Company
Hippocratic AI is building a healthcare-only, safety-focused large language model and AI platform designed to improve patient outcomes.
What you will do
Own end-to-end delivery of audio, voice-conversation, supervised, preference, and RLHF labeling programs through scoping, task design, execution, quality assurance, and delivery.
Translate model training and evaluation needs into labeling tasks, rubrics, success criteria, and release-gating benchmarks.
Source, screen, qualify, onboard, and manage clinicians and specialized raters through marketplaces, vendors, and internal teams.
Write and iterate annotation guidelines for subjective, high-stakes clinical judgment.
Build quality-control systems using gold sets, calibration sessions, spot checks, and inter-rater agreement tracking.
Track throughput, cost, and quality; manage vendor relationships, tooling, dependencies, and timelines across Engineering, Clinical, Infrastructure, ML, and Applied Science.
Requirements
2+ years of technical program management or data operations experience in an AI/ML or data-intensive environment; 3+ years considered for exceptional candidates.
1+ year of hands-on experience with data labeling, annotation, or model evaluation, including writing guidelines and managing quality at scale.
Experience sourcing and managing contractor or expert annotator workforces through marketplaces such as Mercor, Upwork, or Toloka, or through labeling vendors.
Working knowledge of how annotation quality affects model training, including preference data, pairwise comparisons, and rater calibration in RLHF.
Strong understanding of quality metrics including IAA, precision and recall on gold sets, and throughput-versus-accuracy tradeoffs.
Track record of owning critical data or evaluation projects under time pressure, collaborating with ML engineers and applied scientists, and driving results without direct authority.
Comfort with ambiguity in a fast-moving, healthcare-adjacent AI environment.
Nice to have
Experience running or supporting RLHF or other training-data pipelines at an AI lab or applied ML team.
Clinical, healthcare, or other regulated, high-stakes data experience.
Familiarity with Label Studio, Scale AI, Surge AI, or SageMaker Ground Truth.
Experience qualifying specialized raters such as clinicians or bilingual subject-matter experts.
Exposure to reward hacking and specification gaming, and to mitigating them through annotation design.
Culture & Benefits
Full-time onsite role in Menlo Park, California.
Work on a healthcare-only, safety-focused AI platform with direct impact on patient outcomes.
Collaborate with physicians, hospital leaders, AI researchers, ML engineers, applied scientists, and licensed clinicians.
Equal opportunity employer committed to an inclusive team and accommodations during the hiring process.
Recruiting communication comes only from hippocraticai.com email addresses; no payment or sensitive personal information is requested.
Будьте осторожны
Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →