Tech Lead Manager, AI Evaluations

Location: Hybrid, four days a week in the Mountain View, California office

Salary: $250,000–$350,000 annual base salary

Company

Glean develops a Work AI platform with enterprise search, an AI Assistant, AI agents, SaaS connectors, flexible LLM support, and APIs for secure enterprise knowledge and automation.

What you will do

• Own the AI evaluation engineering roadmap and technical direction from defining success criteria through integrating evaluation results into release and product decisions.
• Build scalable, reusable infrastructure for datasets, experiments, graders, trace capture and replay, result analysis, and dashboards.
• Create representative evaluation sets covering common workflows, edge cases, long-tail requests, adversarial inputs, user and domain slices, and historical failures.
• Develop quality measurement systems for correctness, groundedness, retrieval relevance, completeness, instruction following, citations, user preference, and task success.
• Evaluate AI agents across tool selection, arguments, multi-step trajectories, policy compliance, recovery, and final task state.
• Lead and develop a high-performing team of applied AI scientists and engineers while partnering with product, infrastructure, security, and application teams.

Requirements

• 10+ years of experience in software engineering, ML infrastructure, data systems, or a related technical field, including production systems.
• 5+ years of engineering or applied AI team management experience with technical leadership and cross-functional delivery.
• Hands-on experience building or operating evaluation systems for LLM or generative AI products, including model-based evaluators or autograders.
• Strong knowledge of evaluation datasets, benchmarks, deterministic metrics, rubric-based and pairwise judgments, human review, judge calibration, and statistical analysis.
• Strong coding ability in Python or another production language, with experience in APIs, data modeling, testing, reliable distributed systems, and scalable platforms.
• MS or PhD in Computer Science, AI, or a related field.

Nice to have

• Experience with safety testing or red teaming, public benchmarks, cloud infrastructure, Kubernetes, data stores, and observability.
• Experience building experiment pipelines or CI/CD integrations.

Culture & Benefits

• Medical, vision, and dental coverage with generous time off.
• 401k contribution opportunity.
• Home office improvement, education, and wellness stipends.
• Regular company events and daily healthy lunches.
• Inclusive workplace committed to diverse backgrounds and perspectives.

Hiring process

• The interview process includes a brief AI-focused exercise or discussion.