Software Engineer (AI)

TL;DR

Software Engineer (AI): Building the tooling and evaluation infrastructure to accelerate the deployment of high-quality AI features with an accent on scalable eval runners, benchmark curation, and distributed systems. Focus on creating durable datasets and measurement systems to ensure AI quality and reliability at scale.

Location: Hybrid: Must be based in San Francisco, CA or New York, NY (Office attendance required Mon, Tue, Thu)

Salary: $214,000 - $300,000 per year

Company

hirify.global provides a connected workspace that integrates docs, notes, projects, and AI to help teams organize their work and automate tasks.

What you will do

• Develop scalable eval runners and harnesses for local, CI, and scheduled executions.
• Create templates, fixtures, and debugging tools to simplify the addition of high-signal evaluations.
• Build and maintain benchmark and dataset tooling, including curation pipelines and regression tracking.
• Enhance reliability and observability for eval execution, focusing on retries, idempotency, and latency visibility.
• Collaborate with AI product, platform, and infrastructure teams to integrate evals into shipping workflows.

Requirements

• Strong software engineering fundamentals with experience shipping production systems.
• Proficiency with TypeScript/Node and/or Python.
• Experience building reliable distributed systems (queues, retries, idempotency).
• Experience with data pipelines, including batch processing and versioning.
• Practical experience designing measurement or evaluation systems.
• Must be based in or able to work from San Francisco or New York offices.

Nice to have

• Experience building developer tooling such as CLI tools or CI integrations.
• Familiarity with LLM evaluation techniques (rubrics, human review loops, dataset curation).
• Proven track record of collaborating across teams to drive adoption of new workflows.

Culture & Benefits

• Collaborative culture with designated Anchor Days (Mon, Tue, Thu) for in-person work.
• Competitive cash compensation and equity packages.
• Comprehensive benefits package.
• Commitment to diversity, equity, and inclusion in a global employee base.