Subject Matter Expert (AI Quality Evaluation)

Location: Canada or the United States

Company

Develops reliable AI systems, high-quality data, and full-stack technologies for AI applications used by enterprises, governments, and leading technology organizations.

What you will do

• Conduct structured audits of task outputs for accuracy, consistency, and adherence to project guidelines.
• Apply specialized domain expertise to evaluate nuanced or complex work.
• Define and refine quality benchmarks, scoring rubrics, gold standards, and reference materials.
• Analyze audit results, identify systemic quality gaps, and produce actionable quality reports.
• Recommend improvements to workflows, guidelines, and evaluation processes, and support reviewer calibration.

Requirements

• Deep expertise in a specialized field such as medicine, law, or engineering.
• Strong analytical skills for evaluating complex outputs against defined standards.
• Ability to produce clear, structured, and actionable feedback.
• High attention to detail and commitment to accuracy and consistency.
• Experience in quality assurance, auditing, or evaluation frameworks is preferred.

Culture & Benefits

• Part-time, hourly engagement with the ability to set a schedule based on availability.
• Project-based work aligned with audit volume and quality needs.
• Independent work with minimal oversight and expertise-driven deliverables.
• Inclusive and equal opportunity workplace with reasonable accommodations available.