Назад
1 день назад

AI Assurance Engineer - MLE Focus (AI)

187 040 - 438 000$
Формат работы
onsite
Тип работы
fulltime
Грейд
senior
Английский
b2
Страна
US
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
AI Assurance Engineer - MLE Focus (AI/ML): Building evaluation infrastructure, benchmark datasets, and reproducible testing capabilities for recommender and generative AI systems with an accent on statistical rigor, behavioral diagnosis, and safety assurance. Focus on designing adversarial tests, detecting regressions and drift, and integrating metrics, replay workflows, and benchmarks into assurance CI/CD.

Location: San Jose R&D - Security; fully in-person schedule up to 5 days a week.

Salary: $187,040–$438,000 annually, plus potential discretionary bonuses, incentives, and restricted stock units.

Company

TikTok USDS Joint Venture operates data privacy, cybersecurity, trust and safety, and content assurance programs protecting U.S. user data, applications, and algorithms.

What you will do

  • Define reproducible evaluation protocols, benchmark suites, sampling strategies, datasets, statistical analyses, metrics, baselines, and acceptance criteria for recommender and generative AI systems.
  • Build and maintain benchmark and adversarial datasets, including harm-exposure samples, ranking-quality cases, jailbreak prompts, leakage tests, and counterfactual replay inputs.
  • Implement behavioral tests covering safety, ranking quality, fairness, consistency, refusal correctness, factuality within scope, provenance, adversarial robustness, drift, and regressions.
  • Diagnose model-behavior regressions and post-release drift while distinguishing model effects from data, sampling, or infrastructure issues.
  • Integrate benchmarks, metrics, replay workflows, red-team assets, telemetry, and failure-diagnosis capabilities with shared evaluation runtimes and assurance CI/CD.
  • Produce traceable technical findings while maintaining independence from the product teams whose models are evaluated.

Requirements

  • Bachelor’s degree or higher in Computer Science, Machine Learning, Statistics, or a related technical field, or equivalent practical experience.
  • Experience evaluating machine learning systems using clearly defined metrics, datasets, baselines, and reproducible test protocols.
  • Experience developing production-quality evaluation or data-processing software in Python, Go, C++, or a comparable language.
  • Strong knowledge of sampling, hypothesis testing, uncertainty, experiment design, error analysis, and regression diagnosis.

Nice to have

  • Experience with recommender-system evaluation, ranking metrics, counterfactual replay, harm-exposure sampling, cohort consistency, or ranking-quality benchmarks.
  • Experience evaluating generative AI refusal behavior, jailbreak resistance, leakage, factuality within scope, provenance, or multimodal systems.
  • Experience designing red-team tests, adversarial datasets, robustness analyses, or continuous post-release evaluations.
  • Experience integrating benchmarks and metrics with test harnesses, replay pipelines, experiment systems, or CI/CD workflows.
  • Ability to communicate evidence, limitations, and technical findings clearly to product engineering, Trust & Safety, business owners, and governance partners while maintaining ownership boundaries.

Culture & Benefits

  • Fully in-person work up to 5 days per week as the company shifts from a hybrid model.
  • Medical, dental, and vision insurance from day one, plus a 401(k) savings plan with company match.
  • Paid parental leave, short- and long-term disability coverage, life insurance, and wellbeing benefits.
  • 10 paid holidays, 10 paid sick days, and 17 days of Paid Personal Time, prorated upon hire with increasing accruals by tenure.
  • Potential discretionary bonuses, incentives, and restricted stock units; benefits may vary by employment type and country of work.
  • Inclusive, diverse workplace focused on curiosity, humility, resilience, continuous learning, and meaningful impact.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →