Назад
Company hidden
2 дня назад

Member of Technical Staff, Evaluations (Robotics)

180 000 - 300 000$
Формат работы
hybrid
Тип работы
fulltime
Английский
b2
Страна
US
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Member of Technical Staff, Evaluations (Robotics) (Python/C++/simulation): Building benchmarks, evaluation pipelines, and grading systems that measure LLM, VLA, and WAM performance on physical robotics tasks in simulation and on real hardware with an accent on benchmark design, simulation-to-real correlation, and rigorous statistical reporting. Focus on building robot-control harnesses, diagnosing model failures, fine-tuning open-source VLAs, and converting capability gaps into targeted data collection and reinforcement-learning environments.

Location: San Mateo, United States; hybrid

Salary: $180K–$300K plus equity

Company

hirify.global builds data collection systems, operational capabilities, an exabyte-scale data warehouse, and a model evaluation platform for frontier robotics and foundation-model development.

What you will do

  • Design benchmark tasks for LLMs, VLAs, and WAMs covering long-horizon planning, dexterous manipulation, and other physical robotics capabilities.
  • Build verifiers, success detectors, progress metrics, and evaluation harnesses for simulation and real hardware.
  • Develop systems that let LLMs and coding agents control robots directly or plan through VLAs.
  • Measure and improve the correlation between simulation results and real-world performance.
  • Fine-tune open-source VLAs and analyze failures to identify capability gaps and create targeted data-collection and RL environments.
  • Produce partner-facing evaluation results using rigorous statistics and clear reporting.

Requirements

  • Strong programming skills in Python or C++.
  • Experience building benchmarks, evaluation pipelines, or graders for ML models, with careful consideration of what evaluation scores measure.
  • Hands-on experience with robot learning, simulation such as MuJoCo or Isaac Sim, or deploying policies on real hardware.
  • Experience working with LLMs or coding agents and diagnosing model failures from execution traces.
  • An adversarial mindset focused on identifying how metrics could be gamed.
  • Experience building and shipping software in fast-moving environments.

Nice to have

  • Contributions to robotics or agent benchmarks such as LIBERO, BEHAVIOR, RoboArena, or Terminal-Bench.
  • Experience with reinforcement learning, reward modeling, or post-training.
  • Experience with real2sim, system identification, or sim-to-real transfer.
  • Experience applying statistics to evaluation, including confidence intervals and variance across runs.
  • Familiarity with containerized evaluation frameworks such as Harbor.

Culture & Benefits

  • Hybrid work in San Mateo.
  • Full-time employment.
  • Equity offered in addition to the stated salary.
  • Work focused on frontier robotics, foundation models, and high-quality training data.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →