AI Automation Engineer

TL;DR

Senior AI Engineer (AI): Building automated systems to evaluate, test, and enhance conversational AI models with an accent on quality, safety, and reliability. Focus on integrating LLMs into automated evaluation frameworks, automating regression and stress testing, and monitoring model drift, bias, and hallucinations at scale.

Location: Remote

Company

hirify.global is the leading agentic AI platform for enterprise customer experience, backed by WndrCo, Y Combinator, and Index Ventures.

What you will do

• Build Python-based pipelines for automated quality testing of AI responses.
• Integrate LLMs into automated evaluation frameworks and automate regression/stress testing for conversational AI.
• Define evaluation metrics and implement rule-based and AI-driven quality checks.
• Monitor model drift, bias, and hallucinations using automated workflows.
• Embed automated AI evaluation in production by working with APIs, SDKs, and CI/CD pipelines.
• Collaborate with ML engineers, product managers, and QA teams to close the feedback loop.

Requirements

• Strong 4-5 years of experience in Python development (automation, scripting, data handling).
• Experience with LLMs/NLP frameworks.
• Understanding of MLOps / AI deployment pipelines.

Culture & Benefits

• Join an innovative team transforming enterprise customer experience with agentic AI.
• hirify.global is an equal opportunity employer committed to diversity in the workplace.