Senior Automation Engineer (AI)

TL;DR

Senior Automation Engineer (AI): Providing leadership in test automation for AI-driven coaching functionalities, designing and implementing scalable automation frameworks across frontend, backend, and real-time conversational workflows. Focus on advanced API testing for LLM responses, integrating state-of-the-art LLM quality control automation, and championing industry-leading practices for AI-interactive UIs.

Location: Remote

Company

hirify.global is a leader in AI-driven coaching that leverages the latest LLM/ML technologies to deliver personalized, scalable coaching experiences and collects critical business data at scale.

What you will do

• Provide senior-level leadership in test automation for mission-critical AI coaching functionalities.
• Design and implement scalable automation frameworks for Next.js/React frontends, Node.js backends, and conversational workflows.
• Develop advanced automated API testing for RESTful services, microservices, and rigorous validation of non-deterministic LLM responses.
• Lead automated testing of Slack and Microsoft Teams integrations, encompassing bot logic and real-time event processing.
• Architect automated validation suites for WebSocket-based real-time communication and dynamic AI response coherence.
• Integrate state-of-the-art LLM quality control automation, including hallucination detection and bias monitoring.

Requirements

• 8+ years of progressive experience in advanced test automation and quality engineering, with leadership in AI/LLM-driven systems.
• In-depth proficiency testing modern React/Next.js applications using Playwright or Cypress for AI-interactive UIs.
• Expert-level automation of Node.js services, scalable APIs, and microservices utilizing Jest, Mocha, or Supertest.
• Advanced mastery of CI/CD orchestration (GitHub Actions, GitLab CI, Jenkins), containerization (Docker, Kubernetes), and AWS infrastructure.

Nice to have

• Hands-on experience with AI-enhanced automation platforms featuring self-healing and agentic capabilities (e.g., Mabl, testRigor, Applitools).
• Proven expertise with leading LLM evaluation frameworks such as DeepEval, RAGAS, Braintrust, LangSmith, or Arize Phoenix.
• Strong command of Python for test orchestration, data-driven scenarios, and ML model validation.
• Extensive automation experience with Slack/Microsoft Teams bots, webhooks, and WebSocket/event-driven real-time systems.

Culture & Benefits

• Flexible work arrangements.
• Opportunity to work with cutting-edge AI technology.
• Collaborative, diverse team environment.
• Potential meaningful equity opportunity.
• Impactful work with Fortune 500 clients.

Hiring process

• Intro call with Toughbyte.
• 45-min intro & behavioral interview with CTO.
• Test assignment.
• Team interview and final culture call with Hiring Manager + CEO.