Staff Quality Engineer (AI)
Вакансия из Hirify Global, списка международных tech-компанийДля мэтча и отклика нужен Plus
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
TL;DR
Staff Quality Engineer (AI): Designing and implementing quality strategies and evaluation frameworks for non-deterministic AI and agentic systems with an accent on LLM output quality and automated testing. Focus on building scalable test harnesses, establishing AI quality metrics, and mentoring the QE organization.
Location: Must be based in Canada or the United States
Company
AI-powered Marketing Platform that empowers marketers to reach, engage, and convert audiences with precision through programmatic advertising.
What you will do
- Own and drive the end-to-end quality strategy for AI and agentic products, including test architecture and release readiness.
- Design evaluation frameworks for non-deterministic systems, focusing on LLM output quality, agent behavior, and safety testing.
- Develop scalable automated test suites for API, UI, and integration testing, integrating them into the CI/CD pipeline.
- Establish quality benchmarks for accuracy, latency, and hallucination rates to drive data-driven shipping decisions.
- Partner with Engineering, Product, and ML/Data Science teams to align requirements and definitions of done.
- Act as a technical mentor for the QE organization, promoting best practices and improving AI testing company-wide.
Requirements
- 7+ years of professional software quality and test automation experience in complex systems.
- Deep expertise in API and UI automation frameworks such as Playwright, Cypress, Jest, or JUnit.
- Fluency in TypeScript (preferred), JavaScript, Python, or Java.
- Experience testing AI/agentic systems using technologies like OpenAI or Anthropic, including prompt and regression testing.
- Proficiency with CI/CD tools such as Jenkins, GitHub Actions, or CircleCI.
- Strong understanding of scalable system design for web-based APIs and backend processing.
Culture & Benefits
- Competitive salary and retirement options (401K/Pension).
- Comprehensive health and mental health care programs from day one.
- Paid time off packages, including birthdays off.
- Work-from-home reimbursements and optional global WeWork memberships.
- Support for personal development, including budgets for conferences, courses, and books.
- Inclusive and supportive culture with a robust parental leave program.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →