Clinical Quality AI Specialist (Healthcare)

TL;DR

Clinical Quality AI Specialist (Healthcare): Lead validation, governance, and monitoring of clinical AI/LLM tools with an accent on pre-deployment testing, data quality assurance, and patient safety oversight. Focus on building evaluation datasets, conducting red-teaming and prompt testing, refining guardrails, and ensuring URAC/NCQA compliance in virtual care models.

Location: Remote (US-based, geographic zones apply); limited travel (10-15%)

Salary: $128,130–$198,610 + bonus + equity + benefits (PA/NP candidates, zone-dependent)

Company

hirify.global delivers integrated virtual care and navigation to improve access to high-quality healthcare for everyone.

What you will do

• Lead AI clinical tool validation, produce model cards, partner on data quality, build LLM evaluation datasets, run red-teams, test prompts, and assess RAG fidelity.
• Enable clinical-in-the-loop processes by training clinicians, creating SOPs/checklists/escalations, and collaborating with engineering, product, security, and care leaders.
• Establish and administer clinical AI governance including ethics, safety, regulatory compliance, privacy, and change control.
• Define clinical use criteria, human-in-the-loop guardrails, and maintain governance artifacts for audit readiness.
• Support quality and patient safety through QA, URAC/NCQA activities, event detection, RCA/FMEA/CAPA, education, audits, RFPs, and value-based care metrics analysis.
• Conduct assessments of clinical services using continuous improvement methodologies to optimize outcomes, member experience, and cost efficiency.

Requirements

• NP or PA with active, unrestricted license
• 5+ years clinical practice; 2+ years in quality, patient safety, or clinical operations
• Experience with clinical review of LLM outputs, building eval datasets, red-teaming/prompt testing, RAG assessment
• QI/patient safety expertise (PDSA, Lean/Six Sigma, RCA, FMEA, CAPA)
• Working knowledge of URAC, NCQA, CMS; familiarity with HEDIS/MIPS
• Proven quality improvement in value-based care; partnering with analytics/data science
• Excellent documentation and stakeholder communication

Nice to have

• Proficiency in business intelligence and analytics (Tableau/Power BI, basic SQL/statistics)
• Experience in virtual care/care navigation and enterprise audits

Culture & Benefits

• Remote-first culture with work-from-home reimbursement
• Comprehensive medical, vision, dental coverage, disability insurance
• 401(k) savings plan, PTO, DTO, 12 weeks paid parental leave, family building benefits
• Focus on equity, transparency in compensation by geographic zones