Conversational Modelling Research Engineer (AI)

TL;DR

Conversational Modelling Research Engineer (AI): Conduct research on Large Multimodal Models in the context of Conversational Avatars with an accent on modeling verbal and non-verbal aspects of conversation. Focus on developing real-time, low-latency methods for adapting and controlling avatar behavior in expressive, task-specific AudioVisual Multimodal Models.

Location: Preferred San Francisco (hybrid) or London (office opening soon). Remote within the U.S. or Europe available for exceptional candidates.

Company

Series B research lab pioneering AI Humans for real-time human simulation, enabling meaningful face-to-face conversations across languages.

What you will do

• Conduct research on Large Multimodal Models for Conversational Avatars like Neural Avatars and Talking-Heads.
• Develop methods to model verbal and non-verbal conversation aspects with real-time, low-latency adaptation and control.
• Experiment with fine-tuning, adaptation, and conditioning techniques for more expressive, controllable AudioVisual models.
• Partner with Applied ML team to productionize research prototypes.
• Stay current with cutting-edge advancements and contribute to defining future directions.

Requirements

• PhD (or near completion) in relevant field or equivalent research experience.
• Hands-on experience with Large Multimodal Models and generative language models (e.g., VQA, Audio/Video understanding, captioning, behavioral analysis, translation, Speech-to-Speech).
• Experience fine-tuning/adapting VLMs for control, conditioning, or downstream tasks.
• Solid background in deep learning and foundation models.
• Strong PyTorch skills and building deep learning pipelines.

Nice to have

• Knowledge of large-scale model training and optimization.
• Experience in duplex-conversational models.
• Broader understanding of generative AI across modalities.
• Exposure to software development best practices.
• Flexible, experimental mindset across research and engineering.
• Publications at top conferences (EMNLP, COLING, NeurIPS, ICLR, CVPR, ICCV).

Culture & Benefits

• Fast-moving startup environment for experimenting with new ideas and seeing work in production.
• Backed by Sequoia Capital, Y Combinator, Scale Venture Partners.
• Opportunity to shape the future of human-machine interaction.