Engineering Manager, Agent Oversight (AI)

TL;DR

Engineering Manager, Agent Oversight (AI): Building and scaling a platform for deploying, monitoring, evaluating, and improving agentic applications for enterprise and government customers with an accent on agent architectures, ML-driven evaluation, and production infrastructure. Focus on leading multidisciplinary engineering teams, translating customer requirements into platform capabilities, and designing reliable systems that continuously improve agent performance.

Location: United States, with the team based in San Francisco, New York, and Seattle

Salary: $248,400–$310,500 USD base salary, plus equity and benefits

Company

A leading AI data and technology company building reliable AI systems and full-stack platforms for enterprises, governments, generative AI, defense, and autonomous vehicles.

What you will do

• Lead a multidisciplinary team of software and ML engineers delivering capabilities across an AI platform.
• Own the roadmap for agent deployment, monitoring, evaluation, and ML-driven improvement.
• Translate enterprise and government requirements into prioritized and shipped platform capabilities.
• Build and ship features end-to-end, from system design through debugging and testing.
• Drive experimentation based on real customer usage and improve platform performance.
• Establish technical direction, team culture, hiring, mentoring, and scalable engineering processes.

Requirements

• 7+ years of engineering experience, including 2+ years managing engineers or ML engineers responsible for production ML- or LLM-powered systems.
• Hands-on familiarity with agent architectures, including tool use, planning, and multi-agent orchestration.
• Ability to review ML experiment design and evaluation methodology and collaborate credibly with ML engineers and scientists.
• Experience owning platform infrastructure across design, scaling, and adoption by multiple teams.
• Experience working with product managers, forward-deployed engineering teams, and customers to define platform priorities.
• Track record of building and growing high-performing engineering teams through hiring, retention, and improved delivery.

Nice to have

• Experience with evaluation, monitoring, or observability systems for production ML or LLM products.
• Strong understanding of the ML and agent development lifecycle, from experimentation through production iteration.
• Deep knowledge of LLMs, agentic system design, prompt optimization, and integrations with external tools and APIs.
• Research publications, open-source contributions, or patents in agentic systems, LLMs, or applied ML.
• Ability to balance research-driven approaches with pragmatic product constraints in ambiguous environments.

Culture & Benefits

• Inclusive, equal-opportunity workplace focused on reliable AI systems with real-world impact.
• Comprehensive health, dental, and vision coverage.
• Retirement benefits, learning and development stipend, and generous paid time off.
• Equity compensation may be available for eligible roles.
• Additional benefits may include a commuter stipend.

Hiring process

• Salary level and location-specific compensation are determined during the interview process based on experience, qualifications, and interview performance.
• Candidates must wait 90 days before being reconsidered for the same role after an application decision.