TL;DR

AI Engineer (AI): Building the technical infrastructure for the Agent Evaluation Platform, which automatically tests agent performance, manages evaluation datasets, and implements both deterministic and non-deterministic evaluation. Focus on scaling the evaluation systems to handle enterprise workloads and provide reliable insights about agent performance.

Location: Work from anywhere in the world

Company

The Agent Evaluation Platform will automatically evaluate agent performance.

What you will do

Build the technical infrastructure for comprehensive agent evaluation.
Create systems that can automatically test agent performance.
Build tools for managing evaluation datasets.
Implement both deterministic tests and non-deterministic evaluation.
Scale evaluation systems to handle enterprise workloads and provide reliable insights about agent performance.

Requirements

Deep AI engineering experience building AI systems, deploying them in production, and measuring their real-world performance.
Experience with evaluation platforms and understanding of the current limitations of AI testing.
Experience building tools that measure AI system quality and can distinguish between technical functi%hirify_global%lity and user value.
Ability to thrive in uncertainty, build a lot, figure things out on the go, experiment constantly, and handle multiple different tasks across various areas simultaneously.

Culture & Benefits

100% remote work.
Competitive salary in USD + equity.
Budget for tools and learning.
Fast-moving environment with rapid shipping and iteration based on real customer feedback.