Technical Program Manager (AI/ML)

Location: Menlo Park, California, United States; on-site

Company

Hippocratic AI is building a healthcare-only, safety-focused large language model and AI platform designed to improve patient outcomes.

What you will do

• Own end-to-end delivery of audio, voice-conversation, supervised, preference, and RLHF data-labeling programs.
• Translate model-training and evaluation needs into labeling tasks, rubrics, success criteria, and release benchmarks.
• Source, qualify, onboard, and manage clinicians, specialized raters, vendors, and contractor workforces.
• Write and iterate annotation guidelines for subjective and high-stakes clinical judgments.
• Design quality-control systems using gold sets, calibration sessions, spot checks, and inter-rater agreement tracking.
• Track throughput, cost, quality, vendor performance, tooling, dependencies, and delivery timelines across Engineering, Clinical, Infrastructure, ML, and Applied Science.

Requirements

• 5+ years in technical program management or data operations in an AI/ML or data-intensive environment; 3+ years may be considered for exceptional candidates.
• 2+ years of hands-on experience with data labeling, annotation, or model evaluation, including guideline writing and quality management at scale.
• Experience sourcing and managing contractor or expert annotator workforces through marketplaces or labeling vendors.
• Working knowledge of preference data, pairwise comparisons, RLHF, rater calibration, IAA, gold-set precision and recall, and throughput-versus-accuracy trade-offs.
• Experience owning critical data or evaluation projects under time pressure and working with ML engineers and applied scientists without direct authority.
• Comfort operating in ambiguity within a fast-moving, healthcare-adjacent AI environment.

Nice to have

• Experience running or supporting RLHF or other training-data pipelines at an AI lab or applied ML team.
• Clinical, healthcare, regulated, or other high-stakes data experience.
• Familiarity with Label Studio, Scale AI, Surge AI, or SageMaker Ground Truth.
• Experience qualifying specialized raters, including clinicians or bilingual subject-matter experts.
• Exposure to reward hacking and specification gaming, and methods for mitigating them through annotation design.

Culture & Benefits

• Safety-focused healthcare AI work intended to improve patient outcomes.
• Collaboration with physicians, hospital leaders, AI researchers, ML engineers, applied scientists, and licensed clinicians.
• Work on Polaris, a large-scale model system with specialized safety supervisors and evaluation pipelines.
• Equal opportunity workplace with accommodations available during the hiring process.