Prompt Engineer (AI)

TL;DR

Prompt Engineer (AI): Managing end-to-end technical migration workflows for LLM autoraters with an accent on prompt optimization and model performance tuning. Focus on creating baseline prompts, solving edge-case architectures, and measuring quality against gold data sets.

Location: Must be based in the United States

Company

hirify.global provides language, translation, and AI data services to global organizations.

What you will do

• Create baseline prompts for complex parent-child template clusters using Automatic Prompt Generation tools.
• Supervise Automated Prompt Optimization outputs and identify performance deadlocks.
• Manually draft, test, and refine prompts to resolve edge cases and anti-patterns.
• Monitor shadowbot runs to track human and LLM rating disagreements.
• Validate autorater quality by running prompts against gold data to calculate accuracy metrics like F1, precision, and recall.
• Draft technical launch readiness justifications for final certification.

Requirements

• Native fluency in English
• Must be based in the United States
• Bachelor’s, Master’s, or Doctorate degree in Computer Science, Data Science, Computational Linguistics, or a related field.
• At least 2 years of experience as a Prompt Engineer tuning LLMs.
• Strong proficiency in identifying error patterns and analyzing performance using SQL or other data tools.
• Ability to master proprietary tools with minimal supervision.

Nice to have

• Familiarity with enterprise-grade interfaces like the Goose API.
• Hands-on experience with Automated Prompt Optimization systems.
• Understanding of semantics and logic in linguistics.
• Experience in software engineering or AI model evaluation.

Culture & Benefits

• Opportunity to work on large-scale AI service projects.
• Collaborative environment with support for AI-assisted workflows.
• Clear structure for employment eligibility verification and remote work standards.