Prompt Engineer, Agent Prompts & Evals (AI)

TL;DR

Prompt Engineer, Agent Prompts & Evals (AI): Designing, testing, and optimizing AI system prompts and feature-specific prompts for consumer and API products with an accent on shaping Claude's behavior, ensuring model quality, and consistency across product launches. Focus on evaluation development, cross-functional collaboration, and supporting model releases in a fast-paced environment.

Location: Hybrid in San Francisco, CA or New York City, NY. Expected to be in office at least 25% of the time. Visa sponsorship is available.

Salary: $320,000 – $405,000 USD

Company

hirify.global is a public benefit corporation focused on creating reliable, interpretable, and steerable AI systems.

What you will do

• Design, test, and optimize AI system prompts and feature-specific prompts.
• Build and maintain comprehensive evaluation suites for model quality and consistency.
• Partner closely with product, research, and safeguards teams.
• Play a critical role in model releases, ensuring smooth rollouts and catching regressions.
• Help build and improve frameworks and tools for prompt and feature development.
• Mentor product engineers on prompt engineering best practices and evaluation building.

Requirements

• 5+ years of software engineering experience with Python or similar languages.
• Demonstrated experience with LLMs and prompt engineering.
• Strong understanding of evaluation methodologies and metrics for AI systems.
• Excellent written and verbal communication skills.
• Ability to manage multiple concurrent projects and prioritize effectively.
• Experience with version control, CI/CD, and modern software development practices.

Nice to have

• Experience with Claude or other frontier AI models in production settings.
• Background in machine learning, NLP, or related fields.
• Experience with A/B testing and experimentation frameworks.
• Familiarity with AI safety and alignment considerations.
• Experience building tools and infrastructure for ML/AI workflows.

Culture & Benefits

• Competitive compensation and benefits.
• Optional equity donation matching.
• Generous vacation and parental leave, flexible working hours.
• Collaborative group with frequent research discussions.
• Focus on impact and advancing long-term goals of steerable, trustworthy AI.
• Lovely office space to collaborate with colleagues.