Principal Research Scientist - Evaluations (AI)

Location: San Francisco; hybrid work

Salary: $310,000–$420,000 annually

Company

Canva develops design products powered by generative models and serves millions of users worldwide.

What you will do

• Define Canva Research's long-term evaluation strategy across design, image, video, audio, and agentic workflows.
• Build shared evaluation standards covering human preference, learned quality models, MLLM judges, reward signals, and multimodal assessment.
• Establish evaluation gates for launch decisions and diagnose anomalous results, model regressions, and infrastructure artefacts during production training.
• Connect evaluation metrics with user experience, product outcomes, downstream business impact, and editability.
• Partner with Design Generation, Foundation Models, Agents, design, creator, and product teams so evaluation informs training and inference.
• Mentor senior research scientists and engineers and represent Canva's evaluation practice to leadership and the research community.

Requirements

• Track record of defining evaluation frameworks and standards from the ground up and creating measurement systems that influence team decisions.
• Experience linking evaluation metrics to user or business outcomes and diagnosing divergence between them.
• Experience converting subjective human judgement into reliable evaluation signals through rubric design, human-data pipelines, and model training.
• Strong knowledge of multimodal generative models, including diffusion models, transformers, VLMs, and MLLMs.
• Experience with reward modelling, preference learning, or alignment methods involving human feedback.
• Experience setting technical direction across multiple teams in a globally distributed organisation.

Nice to have

• Research background in human perception, psychophysics, aesthetics, or HCI.
• Experience evaluating harm, bias, and safety alongside quality.
• Publications in evaluation, alignment, or generative modelling.
• Background or genuine interest in visual arts and graphic design.

Culture & Benefits

• Flexible hybrid work between home and the office.
• Equity package and health benefits plans.
• 401(k) retirement plan with company contribution.
• Inclusive parental leave for all parents and carers.
• Annual Vibe & Thrive allowance for wellbeing, social connection, and office setup.
• Flexible leave options and virtually conducted interviews.