Высокоэффективная роль с передовым стеком GenAI и RLHF в глобальном масштабе, однако зарплата ниже рыночного медианного значения, а строгий график работы в офисе 5 дней в неделю может стать решающим минусом.
Низкая зарплата
Строгий офис
Передовой стек
Трендовый домен
Четкий scope роли
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Подключить Plus
Описание вакансии
Текст:
Hirify AI/Оригинал
TL;DR
Product Manager, LLM Evaluation & User Insight (E-Commerce Recommendations) (LLM evaluation/e-commerce recommendations): Designing comprehensive automated, model-based, and human evaluation systems for LLM-powered product discovery with an accent on benchmark datasets, quality metrics, and user research. Focus on preference modeling, red-teaming, failure-mode analysis, and translating model insights into improvements for recommendation, search, and AI discovery systems.
Location: Seattle, United States; fully in-person schedule up to 5 days a week
Salary: $141,588–$234,080 annually, plus potential bonuses, incentives, and restricted stock units.
Company
TikTok USDS Joint Venture operates TikTok-related applications with a focus on U.S. user data protection, privacy, cybersecurity, trust, and safety.
What you will do
Design and own automated, model-based, and human evaluation methodologies for LLM capabilities in e-commerce.
Define quality metrics and build standardized benchmark datasets.
Conduct quantitative and qualitative research into buyer and creator needs, mental models, and pain points.
Lead human annotation, preference modeling, pairwise comparisons, RLHF data strategies, and red-teaming initiatives.
Translate LLM quality insights into actionable improvements for Recommendation, Search, and AI Discovery product and engineering teams.
Diagnose model performance and failure modes while recommending improvements to multi-objective ranking and context-aware delivery.
Requirements
Bachelor’s degree or higher.
At least 1 year of experience in Product Management, Data Product Management, or AI Quality/Evaluation.
Hands-on experience or deep understanding of LLM evaluation methodologies and prompt engineering.
Strong quantitative and qualitative analysis, metric design, and ability to translate research and data insights into technical specifications.
Ability to align cross-functional stakeholders, including product managers, ML engineers, data scientists, and operations teams.
Ability to work fully in person in Seattle, up to 5 days per week.
Nice to have
3+ years of experience in product management focused on LLM or GenAI evaluation, AI quality, user research, data annotation strategy, or machine learning applications.
Experience in e-commerce, search and recommendation, or conversational AI.
Familiarity with LLM training and fine-tuning, RLHF, and AI safety procedures.
Culture & Benefits
Medical, dental, and vision insurance from day one.
401(k) savings plan with company match, paid parental leave, disability coverage, and life insurance.
10 paid holidays, 10 paid sick days, and 17 days of paid personal time.
Work environment centered on curiosity, humility, resilience, continuous learning, inclusion, and collaboration.
Будьте осторожны
Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →