Product Validation Engineer II: AI Agent Quality Assurance (QA)

Location: Onsite in Belo Horizonte, Minas Gerais, Brazil at Av Contorno 5800

Company

hirify.global provides software, intellectual property, hardware, and engineering tools for the electronics design chain, from chip design and packaging to boards and complete systems.

What you will do

• Validate AI agent products and customer workflows through structured test planning and execution.
• Test agent behavior across prompts, tool usage, retrieval quality, workflow execution, error handling, and response consistency.
• Evaluate outputs using quality rubrics, benchmarks, regression checks, and metrics including accuracy, relevance, safety, latency, and task success.
• Create test case suites and evaluation datasets covering customer scenarios, edge cases, and domain-specific workflows.
• Document defects, analyze failures, support root-cause triage, and verify fixes through repeatable regression testing.
• Collaborate with product and engineering teams to improve AI agent quality, release readiness, and customer experience.

Requirements

• Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent experience.
• At least 2 years of experience in product validation, software QA, test engineering, software development, or engineering productivity tools.
• Hands-on experience writing test plans and test cases, executing validation cycles, reporting defects, and verifying fixes.
• Familiarity with AI and LLM concepts, AI agents, prompt behavior, RAG workflows, and common AI system failure modes.
• Experience with Python, scripting, or automation for testing, data preparation, log analysis, or reporting.
• Good spoken and written English and Portuguese are required.

Nice to have

• Experience validating AI agents, copilots, conversational products, or workflow automation tools.
• Familiarity with AI evaluation frameworks, benchmarking, red-team testing, or metrics-driven product improvement.
• Experience creating evaluation datasets, golden answers, customer workflow test suites, or regression packs.
• Knowledge of vector databases, embeddings, retrieval evaluation, or agent orchestration frameworks.
• Knowledge of EDA, VLSI design, design verification, emulation, Palladium, or Protium workflows.

Culture & Benefits

• Full-time employment under the Brazilian CLT employment category.
• 40-hour work week.
• Competitive benefits.
• Dynamic, fast-paced collaboration with product and engineering teams around the world.