Senior Prompt Engineer (AI)

TL;DR

Senior Prompt Engineer (AI): Optimizing LLM autoraters through automated and manual prompt generation with an accent on technical migration workflows and template architecture. Focus on analyzing model performance, improving classification accuracy through chain-of-thought techniques, and ensuring rigorous data quality against human baselines.

Location: Must be based in Portugal

Company

A global leader in innovative translation, localization, and data services for AI and machine learning initiatives.

What you will do

• Create baseline prompts for complex parent-child template clusters using automated tools.
• Supervise automated prompt optimization processes and resolve tool deadlocks or performance plateaus.
• Draft and refine manual prompts to handle complex template architectures and edge-case scenarios.
• Monitor shadowbot runs to track discrepancies between human and LLM ratings.
• Validate autorater quality against gold datasets by measuring precision, recall, and F1 scores.
• Prepare technical launch documentation for finalized prompt versions.

Requirements

• Language: Native fluency in Portuguese and fluent English (C1+)
• Location: Must be based in Portugal
• Minimum 4 years of professional experience in prompt engineering or related LLM tuning.
• Strong expertise in chain-of-thought, few-shot learning, and structured LLM outputs.
• Proficiency in SQL and analytical techniques for identifying model error patterns.
• Degree in Computer Science, Data Science, Computational Linguistics, or related field.

Nice to have

• Familiarity with enterprise LLM interfaces such as Goose API.
• Hands-on experience with automated prompt optimization (APO) systems.
• Strong background in linguistic semantics and logic.
• Experience in software engineering or computational linguistics.

Culture & Benefits

• Flexible part-time remote work arrangement.
• Opportunity to work on cutting-edge AI and LLM evaluation projects.
• Collaborative environment focused on advanced prompt engineering and data validation.
• Global exposure to large-scale data service operations.