Назад
Company hidden
3 дня назад

Staff AI Engineer - QA

120 000 - 160 000$
Формат работы
remote (только USA)
Тип работы
fulltime
Грейд
senior
Английский
b2
Страна
US/Canada
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Staff AI Engineer - QA (AI/Healthcare): Building end-to-end AI systems and evaluation infrastructure for healthcare, with an accent on automated testing, clinician-reviewed datasets, statistical accuracy measurement, and safety monitoring. Focus on designing production LLM evaluations, adversarial testing, severity-weighted error metrics, and quality gates for systems affecting patient care.

Location: Remote within the United States or Canada; listed U.S. locations include North Carolina, South Carolina, California, New York, New Mexico, Michigan, Louisiana, Virginia, Utah, Georgia, Texas, Florida, and Tennessee (CST)

Salary: $120,000–$160,000 annually, depending on experience and qualifications.

Company

hirify.global provides mission-critical software solutions for public sector, healthcare, utilities, and private-sector organizations across North America, Europe, Asia, and Australia.

What you will do

  • Build and own the architecture of the AI system end to end, including service layers, data pipelines, and evaluation infrastructure.
  • Design dataset management, automated test harnesses, and historical workflow replay for AI evaluation.
  • Maintain clinician-reviewed evaluation datasets and develop severity-weighted error metrics for clinical and financial impact.
  • Build production monitoring for accuracy, overrides, and safety issues with actionable alerts.
  • Develop adversarial tests for hidden or malicious document instructions and invented content.
  • Partner with product and clinical leads to define accuracy thresholds and criteria for expanding or withdrawing AI capabilities.

Requirements

  • 7+ years of experience in software or machine learning engineering, including at least 3 years focused on AI evaluation, testing, or quality.
  • Experience designing and building complete systems from prototype through production.
  • Experience building and operating production evaluation systems used by engineering teams.
  • At least 2 years evaluating large language model systems in production.
  • Strong statistical skills, including sampling, confidence intervals, and error analysis.
  • 1+ year designing evaluation sets for document extraction, classification, or structured output.
  • Deep knowledge of explainable AI techniques and AI-powered tools such as SHAP, LIME, and Claude.
  • Clear written communication, including design documents, and the ability to explain complex trade-offs to clinicians, engineers, and executives.
  • Curiosity and willingness to adopt emerging AI tools, including copilots, chat-based AI, and automation solutions, in everyday work.

Nice to have

  • Experience in healthcare or another highly regulated industry.
  • Familiarity with metamorphic testing, adversarial testing, or data validation tools such as Great Expectations.
  • Experience mentoring engineers and influencing technical direction across multiple teams.

Culture & Benefits

  • Full-time remote work with flexible work arrangements.
  • Medical, dental, and vision benefits.
  • Three weeks of vacation plus five personal days.
  • Employee stock ownership, RRSP program, and 401(k) matching.
  • Career growth, learning opportunities, community involvement, and a casual work environment.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →