AI QA Trainer (LLM)

TL;DR

AI QA Trainer (LLM): Evaluating and hardening large language models through rigorous testing and safety audits with an accent on hallucination detection, prompt robustness, and chain-of-reasoning reliability. Focus on designing test plans, performing adversarial red-teaming, and documenting failure modes to improve model performance and safety.

Salary: $6–$65 per hour

Company

hirify.global is an organization focused on the evaluation and quality assurance of enterprise-grade AI platforms.

What you will do

• Converse with models on real-world scenarios to verify factual accuracy and logical soundness.
• Design and execute test plans and regression suites to validate model performance.
• Capture reproducible error traces and provide root-cause hypotheses for model failures.
• Suggest improvements to prompt engineering, guardrails, and evaluation metrics.
• Partner on adversarial red-teaming and build dashboards to track quality deltas.
• Develop clear rubrics and pass/fail criteria for model evaluation.

Requirements

• Experience in LLM evaluation, safety testing, or prompt robustness.
• Proficiency in test automation frameworks and tools such as PyTest, OpenAI Evals, or W&B.
• Strong technical skills in Python and SQL for data analysis and automation.
• Ability to perform bias/fairness audits and grounding verification.
• Clear, metacognitive communication skills to document and explain findings.
• Degree in computer science, data science, computational linguistics, or statistics is preferred.

Culture & Benefits

• Fully remote project-based work environment.
• Opportunity to contribute to cutting-edge AI research and development.
• Flexible engagement as a contractor.