Software Engineer, AI Agent & LLM (AI)

TL;DR

Software Engineer, AI Agent & LLM (AI): Building and improving AI systems for a conversational knowledge engine and AI Chat with an accent on user-facing agent experiences, evaluation systems, and shared platforms. Focus on developing robust evaluation datasets, improving task completion, and optimizing the reliability and latency of LLM-powered products.

Location: Mountain View, CA (Hybrid)

Salary: $155,000 – $185,000 USD per year

Company

Leading AI tool for meeting transcription, summarization, and collaboration, transforming meetings into accessible and actionable data.

What you will do

• Build AI-agent experiences for Otter AI Chat to help users reason over conversations and complete complex tasks.
• Advance the conversational knowledge engine via improved knowledge extraction, annotation, and indexing.
• Develop evaluation datasets, automated graders, and release gates to ensure AI quality.
• Diagnose failures across models, prompts, retrieval systems, and backend services.
• Build shared agent infrastructure for orchestration, tracing, debugging, and observability.
• Collaborate with AI, product, and infrastructure teams to deliver end-to-end capabilities.

Requirements

• 3+ years of experience in AI Agent engineering, machine learning, or related fields.
• Strong backend or distributed-systems engineering skills.
• Hands-on experience shipping AI-agent or LLM-powered products.
• Experience evaluating nondeterministic systems and using data/logs to resolve complex failures.
• Ability to take ambiguous problems from initial exploration to production launch.
• Must be based in or able to work in Mountain View, CA.

Nice to have

• Demonstrated ability to use coding agents effectively while maintaining rigorous quality control of output.

Culture & Benefits

• Inclusive and accessible workplace celebrating diversity.
• Comprehensive total rewards package.
• Support for reasonable accommodations throughout the hiring process.