6 дней назад
AI Applied Scientist
225 000 - 280 000$
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
AI Applied Scientist (LLM Evaluation): Building rigorous evaluation systems for an AI shopping agent with an accent on accuracy metrics, benchmarks, experimentation, and LLM judge quality. Focus on designing offline and online evaluation frameworks, fine-tuning and calibrating judge models, and identifying failure modes across retrieval, ranking, recommendations, and conversational understanding.
Location: Remote within the United States
Base salary: $225,000–$280,000 USD per year, depending on skills, experience, role level, geographic location, and scope.
Company
develops an AI shopping agent that finds products across the web with a focus on accuracy, quality, and trust.
What you will do
- Define and evolve accuracy metrics across retrieval, ranking, recommendations, and shopping outcomes.
- Design experiments to measure model and product improvements or regressions.
- Build and maintain evaluation datasets, benchmarks, scoring frameworks, and automated reporting.
- Improve LLM judges through prompting, calibration, and fine-tuning.
- Translate ambiguous product questions into measurable hypotheses and analyses.
- Partner with ML Engineering, AI Engineering, and Product to validate model changes and identify failure modes.
Requirements
- 5+ years of experience in Applied ML, AI Research, or Applied Science; PhD or equivalent depth is strongly preferred.
- Hands-on experience evaluating modern AI/ML systems, including LLMs, agents, ranking, or recommendation systems.
- Direct experience with LLM-based systems, such as judge models, RAG, prompt engineering, fine-tuning, or RLHF.
- Strong foundations in experimentation, A/B testing, causal inference, and statistical rigor.
- Ability to define ambiguous problems and communicate clearly across ML, engineering, and product.
- Must be able to work remotely from the United States.
Culture & Benefits
- Fully remote work within the United States.
- Equity through stock options.
- Medical, dental, and vision coverage.
- 401(k) plan.
- Flexible PTO, company holidays, periodic offsites, and team gatherings.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →
Похожие вакансии
Anthropic
21 час назад
Data Scientist, Product (AI)
285 000 - 380 000$
6 дней назад
Senior Staff Data Scientist
240 000 - 249 500$
4 дня назад
People Analytics Data Scientist (AI)
136 400 - 272 500$
Datadog
5 дней назад
Staff Applied Scientist (AI Dashboards)
276 000 - 345 000$
6 дней назад
Senior Applied Scientist (LLM Code Agents)
192 200 - 260 000$
5 дней назад
Applied Scientist (AI & Machine Learning)
175 000 - 200 000$