Labeling & Evaluation Agent (French) (AI)

Location: On-site in Lisbon, Portugal

Company

hirify.global is supporting a leading technology company specialising in artificial intelligence and the development and continuous improvement of AI models.

What you will do

• Evaluate AI-generated responses using detailed quality assessment and annotation guidelines.
• Label and annotate language model outputs accurately and consistently.
• Assess content for quality, relevance, factual accuracy, safety, and policy compliance.
• Apply complex annotation guidelines across different scenarios and content types.
• Identify inconsistencies, edge cases, and potential issues, escalating them when necessary.
• Analyse structured text, language data, and AI-generated content while providing actionable feedback for AI model improvement.

Requirements

• Native-level proficiency in French.
• English proficiency at B2 level or above.
• Strong understanding of Generative AI, large language models (LLMs), and AI annotation or evaluation platforms.
• Excellent analytical and critical thinking skills with strong attention to detail.
• Ability to make objective, evidence-based decisions and consistently apply complex written guidelines and policies.
• Strong reading comprehension and written language skills.
• Ability to evaluate structured text, datasets, and AI-generated content.
• Ability to meet productivity, accuracy, and quality targets.
• Professionalism, integrity, and discretion when handling confidential information.

Culture & Benefits

• Full-time employment in Lisbon at a modern city-centre office with open spaces and public transport access.
• Comprehensive initial training, guidance, coaching, and mentoring opportunities.
• 24/7 rotational working schedule with two rotational days off.
• International and multicultural working environment.
• Health and wellbeing initiatives and a monthly meal allowance.
• Opportunity to work in the rapidly developing field of Generative AI and large language models.