Model Evaluation Lead (Embodied AI)

Location: Redwood City, California, United States; onsite

Salary: $140K–$160K annually, plus equity.

Company

Builds general-purpose robots powered by a proprietary embodied AI foundation model and deployed with customers across multiple industries.

What you will do

• Lead, manage, and develop the model evaluation team, including priorities, coaching, performance, onboarding, and training.
• Translate research questions into evaluation plans with hypotheses, success criteria, deliverables, and checkpoint comparisons.
• Design task definitions, scoring rubrics, test scenarios, trial plans, and measurements covering task success, quality, cycle time, interventions, and recovery behavior.
• Standardize evaluation setup, resets, scoring, intervention rules, traceability, and coordination with hardware and operations partners.
• Review robot behavior, logs, and video to distinguish model failures from hardware, software, setup, or execution issues and identify recurring failure patterns.
• Own regression coverage, reporting, benchmarks, held-out scenarios, evaluation tooling, automation, and dashboards that inform model improvements and deployment decisions.

Requirements

• Experience in robotics, machine learning, applied research, or model evaluation, with direct ownership of evaluation design and analysis.
• Experience managing and developing a technical team, including setting priorities, giving feedback, and establishing effective working practices.
• Strong experimental judgment, including defining success, controlling relevant variables, identifying test-coverage gaps, and assessing whether improvements are meaningful.
• Technical fluency working with researchers on model checkpoints, generalization, data quality, and experimental tradeoffs.
• Comfort using Python or similar tools to analyze results, inspect logs, and improve evaluation workflows.
• Clear communication and strong ownership, including turning ambiguous requests into executable plans and explaining findings to research and operations teams.

Nice to have

• Experience evaluating robot manipulation policies, embodied AI, autonomous systems, or other machine learning systems operating in the physical world.
• Experience building regression suites, benchmark datasets, failure taxonomies, or evaluation tools and connecting findings to model development or deployment readiness.

Culture & Benefits

• Full-time onsite work in Redwood City, United States.
• Technical rigor, mutual respect, diversity, and equal employment opportunity.
• Equity offered as part of compensation.
• Applications are encouraged from candidates who do not meet every listed criterion but bring relevant problem-solving ability and grit.