Senior Manager, AI Performance & Operations (AI Engineering)

TL;DR

Senior Manager, AI Performance & Operations (AI Engineering): Owns the production performance and continuous improvement of Devoted’s core agentic workflows for service and experience with an accent on clear performance thresholds, reliable monitoring, and explicit fallback paths. Focus on early detection of degradation, diagnosing the root cause, and driving corrective action to improve outcomes.

Location: Remote (USA)

Salary: $110,000 - $172,000 / year

Company

hirify.global is on a mission to dramatically improve the health and well-being of older Americans by delivering a complete and integrated healthcare solution.

What you will do

• Own the health of major voice and text AI workflows, ensuring business, quality, safety, and operational outcomes are met.
• Maintain a current view of all major workflows, including status, maturity, failure modes, and next actions.
• Monitor weekly indicators and oversee production dashboards to track behavior independently.
• Lead response for AI regressions and drive post-mortems to closure.
• Run structured cycles for mini-launches like model swaps, prompt changes, and tool updates.
• Guide new workflows during incubation to ensure they launch with proper instrumentation, eval coverage, and safety models.

Requirements

• Applied AI Expertise: Strong production experience with LLMs, RAG, voice AI, agentic orchestration, or equivalent ML systems.
• Demonstrated experience designing or operating rubrics, scoring frameworks, regression tests, and production-sampling workflows.
• A clear mental model for distinguishing failure modes, such as model reasoning, tool failure, retrieval issues, and workflow logic.
• Comfort with active experimentation—including prompt/context tuning, retrieval redesign, and safe model migrations—beyond simple issue triage.
• Excellent cross-functional judgment and the ability to communicate clearly in ambiguous, high-accountability environments.
• Deep personal accountability with a drive to own results rather than just producing recommendations.

Nice to have

• Experience in healthcare, regulated workflows, contact center operations, or high-stakes service quality/reliability functions.

Culture & Benefits

• Employer sponsored health, dental and vision plan with low or no premium.
• Generous paid time off.
• $100 monthly mobile or internet stipend.
• Stock options for all employees.
• Bonus eligibility for all roles excluding Director and above; Commission eligibility for Sales roles.
• Parental leave program and 401K program.