AI Quality & Validation Specialist (AI)

TL;DR

AI Quality & Validation Specialist (AI): Testing and validating AI-enabled solutions and applications with an accent on AI output accuracy, prompt validation, and user-centric testing. Focus on identifying anomalies in generated content, documenting software defects, and improving the overall quality of AI-driven outputs.

Location: Hybrid in Annandale, VA

Company

A service provider focused on cybersecurity, development, and IT infrastructure for the federal government.

What you will do

• Perform functional, regression, and user-focused testing of AI-enabled features prior to release.
• Evaluate AI-generated outputs for quality, relevance, accuracy, and alignment with prompts.
• Identify, document, and track software defects, content issues, and output anomalies.
• Conduct scenario-based testing from an end-user perspective to validate workflows.
• Collaborate with developers and engineers to clarify expected behavior and resolve issues.
• Contribute to the development of repeatable validation procedures, checklists, and quality review practices.

Requirements

• 1–3 years of experience in software testing, QA support, or AI output evaluation.
• Associate’s or Bachelor’s Degree in a related field.
• Strong attention to detail with the ability to identify inconsistencies and defects.
• Strong written and verbal communication skills.
• Ability to follow structured instructions and evaluation criteria.
• Must be based in or be able to work in a hybrid setup in Annandale, VA.

Nice to have

• Experience evaluating generative AI outputs (text, images, audio, or video).
• Familiarity with defect tracking tools, test case management, or Agile environments.
• Experience in a federal contracting environment.
• Active security clearance or eligibility to obtain one.

Culture & Benefits

• Comprehensive health coverage with three medical plan choices, dental, and vision insurance.
• 401k with competitive matching and vesting schedule.
• Paid Time Off and eleven federal holidays.
• Parental leave and continuing education assistance.
• Wellness perks including Calm Health app and WellHub gym subsidy.
• Various insurance options: life, AD&D, short-term and long-term disability, and pet insurance.