Назад
Company hidden
9 дней назад

Analyst, Applied Validations (AI)

200 000 - 300 000$
Формат работы
onsite
Тип работы
fulltime
Грейд
middle
Английский
b2
Страна
US
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Analyst, Applied Validations (AI): Designing and running customer validations and backtests that compare simulated results with real-world ground truth, with an accent on statistical judgment, reproducible experimentation, and model evaluation. Focus on diagnosing gaps across segments and populations, building Python and API-driven workflows, and translating evidence into improvements for Modeling, Evals, Data, and Product.

Location: Onsite in New York or San Francisco, United States

Salary: $200,000–$300,000 USD per year, plus equity and comprehensive benefits.

Company

hirify.global develops an AI simulation of society with generative agents and a foundation model designed to predict human behavior.

What you will do

  • Scope customer validation and backtest studies, selecting use cases, metrics, success criteria, and appropriate validation approaches.
  • Adapt customer survey instruments for apples-to-apples replication in hirify.global and document all changes.
  • Run simulations and compare results with human ground truth using TVD, Spearman rank correlation, and top-K/bottom-K agreement for MaxDiff.
  • Diagnose gaps by segment and population, distinguishing model, agent-bank, ground-truth, coverage, and methodology issues.
  • Translate validation findings into reproducible examples and recommendations for Modeling, Evals, Data, and Product, then retest improvements.
  • Build reusable Python pipelines, API-driven experiments, and LLM skills, and present clear customer readouts to technical and business stakeholders.

Requirements

  • Solid statistical foundation covering sampling, variability, correlation, bias, data quality, population differences, and evidence-based conclusions.
  • Ability to learn unfamiliar methods and tools quickly, investigate unexpected results, and improve approaches through structured experimentation.
  • Ability to use Python and APIs for data analysis and reproducible experiments, including Claude Code, Codex, or similar coding agents, and to validate their outputs.
  • Ability to communicate analytical results, limitations, and implications clearly to technical and business audiences and connect findings to customer priorities.
  • Ability to turn findings into actionable recommendations, collaborate with Modeling, Evals, Data, and Product, and document lessons for others.

Nice to have

  • Experience with backtesting, predictive-model evaluation, or iterative performance improvement against a target metric.
  • Familiarity with holdout datasets, data leakage, and generalization testing.
  • Familiarity with survey design, panel sampling, TVD, MaxDiff, and stated-intent versus observed-behavior differences.
  • Background in market research, consumer insights, survey science, behavioral research, or research-agency analytics.
  • Experience with Qualtrics or similar survey platforms and panel providers.
  • Experience evaluating LLM outputs or building reusable pipelines, LLM skills, and automations.
  • Experience in solutions engineering, applied data science, technical consulting, or presenting analytical findings to customers.

Culture & Benefits

  • Hands-on individual contributor role combining coding, data investigation, and customer-paced experimentation.
  • Collaboration across Solutions Consulting, Deployments, Modeling, Evals, Data, and Product.
  • Competitive compensation including equity for eligible roles, subject to board approval.
  • Comprehensive medical, dental, and vision coverage.
  • Flexible time off policies supporting work-life balance.
  • Equal-opportunity workplace with reasonable accommodations available during the application process.

Hiring process

  • The process includes thoughtful conversations and clear examples of past work to align on fit, working style, and expectations.
  • Candidates may reapply for the same role after a 90-day waiting period.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →