Agent Evaluation and Verification Engineer

Location: San Francisco, United States; on-site

Salary: $220,000–$300,000 annually for L5 Senior; L6 compensation includes $220,000–$300,000 base with $270,000–$350,000 equity. Additional equity is offered on employee-friendly terms.

Company

Wispr Flow is an AI research and product company building voice interfaces for computing, including Flow and Notetaker.

What you will do

• Build end-to-end verification loops that allow AI agents to determine when their work is complete.
• Create reliable testing environments and convert failures and human feedback into durable evaluation signals.
• Improve the delivery of context, instructions, skills, and memory to agents, and measure their impact on performance.
• Operate the agent fleet as a production system and measure quality, adoption, and impact.

Requirements

• Experience building internal tools adopted by other engineers and measuring their impact.
• Deep hands-on use of agentic coding tools, with clear opinions about their strengths and limitations.
• Experience building evaluation or verification infrastructure such as test harnesses, CI systems, evaluation pipelines, or benchmarks.
• Ability to work across application code, infrastructure, and unfamiliar systems.
• Empirical approach to engineering, attention to subtle failure modes, and comfort owning ambiguous problems.
• On-site work in San Francisco, United States.

Nice to have

• Prior experience in voice, model training, or related product domains is not required but may be relevant.

Culture & Benefits

• Work on AI products intended to improve how people interact with computers.
• Equity includes extended exercise windows and a combination of ISOs and RSUs.
• Visa sponsorship is available for H1B, O1, EB1, L1, STEM OPT, and other categories, subject to role-specific feasibility.
• Commitment to a fair and accessible interview process with accommodations available.