Evaluation Researcher (AI)

Location: San Francisco, California, United States — onsite

Salary: $180,000–$210,000 per year

Company

Well-funded early-stage frontier AI startup developing multimodal AI, real-time voice systems, intelligent agents, and AI companions.

What you will do

• Develop evaluation frameworks for real-time, multimodal AI systems.
• Design benchmarks and testing methodologies for speech, reasoning, interaction, and conversational behaviour.
• Measure and analyse model performance and behaviour.
• Design experiments and translate findings into improvements to AI systems.
• Help evaluate AI companions that understand live environments, manage interruptions, and interact with users in real time.

Requirements

• Strong machine learning research and engineering fundamentals.
• Experience with LLM evaluations, benchmarking, or model assessment.
• Strong Python and deep learning experience.
• Familiarity with multimodal models, speech AI, or conversational agents.
• Ability to design experiments, analyse model behaviour, and communicate findings as system improvements.

Culture & Benefits

• Work onsite in San Francisco as part of an early-stage research team.
• Opportunity to work on frontier AI systems backed by leading venture capital investors and prominent startup accelerators.
• Focus on next-generation multimodal, voice, and intelligent-agent technologies.