Senior Manager, Machine Learning (Data Operations)

TL;DR

Senior Manager, Machine Learning (Data Operations): Leading labeling quality and methodology across the organization with an accent on designing annotation tasks, defining quality frameworks, and managing outsourced labeling vendors. Focus on ensuring labeled datasets meet production standards for ML models through rigorous measurement, platform management, and cross-functional collaboration.

Location: Must be based in Canada

Salary: $123,100–$186,800/year (based on location)

Company

hirify.global is the world's first Active Insurance provider, combining comprehensive insurance coverage with innovative cybersecurity tools to help businesses manage and mitigate digital risk.

What you will do

• Define annotation guidelines, taxonomies, and edge-case protocols for all labeling programs.
• Manage the Label Studio platform, driving requirements and workflow designs in partnership with data engineering.
• Oversee outsourced labeling vendors, setting quality SLAs and ensuring accuracy and throughput targets are met.
• Collaborate with ML engineers and product managers to translate model requirements into structured labeling tasks.
• Track operational metrics like label accuracy and IAA scores to drive continuous improvement and cost efficiency.

Requirements

• Must be based in Canada
• 5+ years of experience in ML data operations, data labeling, or related engineering fields.
• Deep expertise in annotation quality frameworks including IAA, gold standard evaluation, and error taxonomy.
• Proven track record managing outsourced labeling vendors or BPOs for ML data production.
• Proficiency in Python and SQL, with experience building tooling for labeling workflows.
• Direct experience managing labeling platforms such as Label Studio, Scale AI, or Labelbox.

Culture & Benefits

• 100% medical, dental, and vision coverage.
• Flexible PTO policy.
• Annual home office stipend and WeWork access.
• Access to mental and physical health wellness programs like Headspace and Lumino.
• Remote-first, inclusive culture focused on shared ownership and protecting businesses from digital risk.