Labeling & Evaluation Agent (German)

Location: On-site in Lisbon, Portugal

Company

hirify.global is recruiting for a leading technology company specialising in artificial intelligence.

What you will do

• Evaluate AI-generated responses, including chatbot outputs, against quality, relevance, factual accuracy, safety, and policy compliance criteria.
• Label and annotate language-model outputs accurately and consistently across different scenarios and content types.
• Analyse structured text, language data, datasets, and AI-generated content.
• Identify inconsistencies, edge cases, and potential issues, escalating them when necessary.
• Provide accurate evaluations, annotations, and actionable feedback to support continuous AI model improvement.
• Maintain confidentiality and comply with project standards and data-security requirements.

Requirements

• Native-level proficiency in German.
• English proficiency at B2 level or above.
• Strong understanding of Generative AI, large language models (LLMs), and AI annotation or evaluation platforms.
• Excellent analytical and critical-thinking skills with strong attention to detail.
• Ability to make objective, evidence-based decisions and consistently apply complex written guidelines and policies.
• Strong reading comprehension and written-language skills.
• Ability to evaluate structured text, datasets, and AI-generated content.
• Ability to meet productivity, accuracy, and quality targets.
• Professionalism, integrity, and discretion when handling confidential information.

Culture & Benefits

• Full-time on-site employment in Lisbon with a modern city-centre office and public-transport access.
• Comprehensive initial training, guidance, coaching, and mentoring opportunities.
• 24/7 rotational working schedule with two rotational days off.
• International and multicultural working environment.
• Health and wellbeing initiatives and a monthly meal allowance.
• Opportunity to work in the rapidly developing field of Generative AI and large language models.