Вакансия из Hirify Global, списка международных tech-компаний Для мэтча и отклика нужен Plus
Пожаловаться
Зарплата и рынок
развернуть
$140к-200к/год
$180к$210кмедиана$280к
По похожим ролям (50) ~ $210к/год
На СНГ рынке: $3.9к/мес ($3.2к - $5.8к)
86
Сильная возможность
развернуть
У роли четкая ответственность за качество оценки в быстро развивающейся сфере ИИ и робототехники, а обязанности дают возможность влиять на исследования и поставку данных.
Четкая зона ответственности
Актуальные технологии
Перспективная сфера
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Подключить Plus
Описание вакансии
Текст:
Hirify AI/Оригинал
TL;DR
Member of Technical Staff, Evals & Data Quality (AI): Building benchmark methodologies, audit rubrics, automated graders, and quality standards for robotics and embodied AI data with an accent on machine learning evaluation, statistical measurement, and data quality. Focus on designing ground-truth pipelines, detecting metric gaming, reducing manual review costs, and making evidence-based ship or no-ship decisions.
Location: Hybrid in New York, Toronto GTA, or San Francisco
Salary: Toronto: CA$140K–CA$200K; New York and San Francisco: $140K–$200K
Company
hirify.global builds data infrastructure for robotics and embodied AI, including data capture, labeling, and hardware-enabled workflows used to train and validate humanoid and embodied AI systems.
What you will do
Own benchmark methodology, including evaluation sets, sampling, metrics, versioning, and comparisons.
Write customer quality specifications and audit rubrics aligned with customer rejection criteria.
Run human audits and coordinate with annotation and operations on sampling, agreement, throughput, and cost.
Build automated checks and graders that reduce manual review costs and resist gaming.
Track lost recorded hours, report quality results, and make ship or no-ship decisions on data.
Requirements
Experience in evaluation, measurement, or data quality for machine learning systems, or in data operations at scale.
Machine learning and data science background with strong Python skills.
Working knowledge of statistics, including sampling, precision and recall, agreement, and confidence intervals.
Experience designing evaluation protocols and ground-truth pipelines from scratch.
Ability to challenge research teams’ results and support challenges with evidence.
Ability to identify how metrics can be gamed and close the resulting gaps.
Nice to have
Experience with 3D pose, trajectory, and reconstruction metrics such as MPJPE and ATE.
Knowledge of sampling design, inter-annotator agreement, loss accounting, and managing annotators or vendors.
Experience with verifiers, learned scoring, active learning, and human-in-the-loop evaluation.
Experience with quality scoring, filtering, and composition of large datasets.
Culture & Benefits
Inclusive work environment welcoming applicants from all backgrounds.
Equal-opportunity hiring and accommodations support during recruitment.
AI-assisted tools may support application evaluation, with final consideration remaining subject to human judgment.
Будьте осторожны
Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →