Sdet- II (AI)

TL;DR

SDET-II (AI): Leading end-to-end qualification lifecycle for AI/LLM systems, designing and implementing scalable automated test suites, and enhancing frameworks to continuously improve complex AI and LLM workflows with an accent on measuring factual accuracy, hallucination rates, and overall model reliability. Focus on developing monitoring and alerting systems to track LLM production quality, safety, and performance in real time, as well as championing quality engineering practices across the organization.

Location: On-site in Pune, Maharashtra

Company

hirify.global is the market-leading revenue productivity platform that combines on-the-job learning and deal execution to get more revenue per rep.

What you will do

• Own the end-to-end qualification lifecycle for AI/LLM systems from ideation and implementation to CI/CD integration.
• Design and implement scalable automated test suites across unit, integration, regression, and system levels.
• Develop evaluation pipelines to measure factual accuracy, hallucination rates, bias, robustness, and overall model reliability.
• Develop monitoring and alerting systems to track LLM production quality, safety, and performance in real time.
• Mentor junior engineers in AI testing, automation, and quality best practices.
• Champion quality engineering practices across the organization, ensuring delivery meets business goals, user experience, cost of operations etc.

Requirements

• Bachelor’s or Master’s in Computer Science, Engineering, or equivalent.
• 4+ years in software development, SDET, or QA automation.
• Proficiency in GoLang, Java, or Python.
• Proven ability to design CI/CD pipelines with automated regression and evaluation testing.
• Hands-on exposure to LLMs, GenAI applications.
• 2+ years of hands-on experience with LLM APIs and frameworks (OpenAI, Anthropic, Hugging Face).

Nice to have

• LLM testing & evaluation tools: MaximAI, OpenAI Evals, TruLens, Promptfoo, LangSmith
• Building LLM-powered apps: prompt pipelines, embeddings, RAG, AI workflows
• CI/CD design for application + LLM testing

Culture & Benefits

• hirify.global is proud to be an Equal Opportunity Employer.
• Culture reflects employee's globally diverse backgrounds along with a commitment to customers, and each other, and a passion for excellence.
• Live up to values: DAB, Delight your customers, Act as a Founder, and Better Together.