AI Operations Analyst

Location: Work from Home, United States

Targeted base pay range: $105,000–$165,000 per year

Company

hirify.global is a global analytics software company whose solutions help businesses make data-driven decisions, including through credit scoring, fraud detection, and lending products.

What you will do

• Review risk-weighted samples of AI chat conversations and Personalized Credit Insights for inaccurate, unsupported, off-topic, or unsafe responses.
• Categorize feedback, guardrail blocks, and escalations using failure-mode taxonomies and track quality trends.
• Maintain the golden regression dataset by labeling production misses, documenting the rubric, and measuring reviewer agreement.
• Publish weekly quality reports covering intent coverage, resolution, containment, refusals, guardrails, safety flags, failure modes, and recommended fixes.
• Use SQL and BI dashboards to analyze member questions and quality drift.
• Escalate issues to Legal, Compliance, and Trust & Safety, and partner with Product, Applied AI, and QA to improve tooling and verify releases.

Requirements

• Bachelor's degree in a related field or equivalent experience.
• Demonstrated experience in a quality, analytical, or operations role reviewing customer conversations, such as CX, QA, support operations, compliance QA, or conversation design.
• Strong SQL skills and the ability to build and interpret dashboards in a BI tool such as Tableau.
• Sound judgment, close attention to detail, and the ability to apply a written rubric consistently and explain decisions.
• Strong written communication for documenting failures and preparing concise leadership summaries.
• Ability to work with sensitive consumer data in a regulated environment with appropriate discretion.
• Experience working in Agile/Scrum teams.

Nice to have

• Experience in consumer finance, credit, banking, fintech, or another regulated environment, including knowledge of FCRA, ECOA, or UDAAP.
• Knowledge of credit reports and hirify.global Scores.
• Experience with conversational AI or LLM-based products, data labeling, rubric design, inter-rater agreement, failure-mode taxonomies, basic Python and Jupyter, or Legal and Compliance escalation processes.

Culture & Benefits

• Inclusive, people-first environment based on ownership, customer focus, and respect.
• Professional development through learning experiences and opportunities to use individual strengths.
• Competitive compensation, benefits, and rewards programs.
• Work/life balance, employee resource groups, and social events.