Technical Program Manager (AI)
ΠΡΡΡ & Π‘ΠΎΠΏΡΠΎΠ²ΠΎΠ΄
ΠΠ»Ρ ΠΌΡΡΡΠ° Ρ ΡΡΠΎΠΉ Π²Π°ΠΊΠ°Π½ΡΠΈΠ΅ΠΉ Π½ΡΠΆΠ΅Π½ Plus
ΠΠΏΠΈΡΠ°Π½ΠΈΠ΅ Π²Π°ΠΊΠ°Π½ΡΠΈΠΈ
TL;DR
Technical Program Manager (AI): Managing high-priority frontier model evaluation and research programs with an accent on designing benchmarks and coordinating human data campaigns. Focus on translating ambiguous research questions into concrete execution plans and performing hands-on technical analysis using Python and SQL.
Location: Must be based in San Francisco, CA (Hybrid: 3 days in office). Relocation assistance available.
Salary: $207,000 β $230,000 USD + Equity
Company
is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity.
What you will do
- Manage frontier evaluation projects from initial research questions to delivered benchmarks.
- Translate ambiguous model capability questions into concrete eval designs, success metrics, and execution plans.
- Design and manage human data campaigns, including task design, trainer instructions, and quality control workflows.
- Perform hands-on technical work including prompt iteration, model-based evaluation workflows, and lightweight scripting.
- Build roadmaps and operating rhythms to keep fast-moving research efforts aligned and unblocked.
- Coordinate across research, engineering, product, safety, legal, and external vendors.
Requirements
- Experience in technical program management, research operations, or data operations.
- Proficiency in Python and SQL to analyze datasets, inspect model outputs, and automate workflows.
- Strong understanding of large language models (LLMs), including prompting, evaluation, and failure modes.
- Ability to function as both an individual contributor (IC) and a program manager.
- Must be based in San Francisco, CA, with a requirement of 3 days per week in the office.
Culture & Benefits
- Competitive compensation with equity offers.
- Relocation assistance provided for new employees.
- Opportunity to work on the most advanced AI models and advance the science of model evaluation.
- Inclusive work environment valuing diverse perspectives and experiences.
ΠΡΠ΄ΡΡΠ΅ ΠΎΡΡΠΎΡΠΎΠΆΠ½Ρ: Π΅ΡΠ»ΠΈ ΡΠ°Π±ΠΎΡΠΎΠ΄Π°ΡΠ΅Π»Ρ ΠΏΡΠΎΡΠΈΡ Π²ΠΎΠΉΡΠΈ Π² ΠΈΡ ΡΠΈΡΡΠ΅ΠΌΡ, ΠΈΡΠΏΠΎΠ»ΡΠ·ΡΡ iCloud/Google, ΠΏΡΠΈΡΠ»Π°ΡΡ ΠΊΠΎΠ΄/ΠΏΠ°ΡΠΎΠ»Ρ, Π·Π°ΠΏΡΡΡΠΈΡΡ ΠΊΠΎΠ΄/ΠΠ, Π½Π΅ Π΄Π΅Π»Π°ΠΉΡΠ΅ ΡΡΠΎΠ³ΠΎ - ΡΡΠΎ ΠΌΠΎΡΠ΅Π½Π½ΠΈΠΊΠΈ. ΠΠ±ΡΠ·Π°ΡΠ΅Π»ΡΠ½ΠΎ ΠΆΠΌΠΈΡΠ΅ "ΠΠΎΠΆΠ°Π»ΠΎΠ²Π°ΡΡΡΡ" ΠΈΠ»ΠΈ ΠΏΠΈΡΠΈΡΠ΅ Π² ΠΏΠΎΠ΄Π΄Π΅ΡΠΆΠΊΡ. ΠΠΎΠ΄ΡΠΎΠ±Π½Π΅Π΅ Π² Π³Π°ΠΉΠ΄Π΅ β