7 дней назад
AI Engineer
163 800 - 306 000$
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
AI Engineer (LLM Evaluation): Building reliable AI-native product behavior across prompting, retrieval, routing, tool use, and evaluation systems with an accent on non-deterministic model behavior, statistical quality measurement, and production safety. Focus on detecting regressions and hallucinations, designing guardrails and fallbacks, and balancing quality, cost, latency, and robustness across real user workflows.
Location: Hybrid work location; employment is currently available in the United States, with specific roles available in the UK
Salary: $163,800–$306,000 per year for candidates based in the United States, plus possible equity or commission depending on the position.
Company
builds a platform for creating and governing internal software connected to business data, enterprise policies, shared components, and AI tools.
What you will do
- Own the quality, safety, variance, and failure modes of AI-powered features across product surfaces.
- Design prompting, retrieval, routing, and tool-use strategies for non-deterministic systems.
- Build evaluation systems using statistical signals, distributions, trends, golden datasets, and regression detection.
- Diagnose and resolve hallucinations, partial correctness, instruction drift, and context sensitivity.
- Implement guardrails, fallbacks, and degradation paths that preserve product usefulness.
- Collaborate with product and infrastructure teams to balance quality, cost, latency, and robustness before shipping.
Requirements
- 6+ years of professional engineering experience owning complex production systems.
- Experience owning AI or LLM behavior beyond basic API integration, including variance and failure-mode mitigation.
- Ability to reason about probabilistic systems and tradeoffs such as quality versus cost and speed versus robustness.
- Experience designing or maintaining evaluation frameworks, regression detection, golden datasets, or human-in-the-loop feedback loops.
- Strong product judgment and the ability to work independently in ambiguous problem spaces.
- Experience working across technologies such as TypeScript, Node.js, and React.
Nice to have
- Production experience with RAG, agentic systems, or tool-using models.
- Familiarity with vector databases, embeddings, or retrieval pipelines.
- Exposure to fine-tuning, model routing, or post-training techniques.
- Experience building shared AI infrastructure or mentoring engineers on evaluation-driven development.
Culture & Benefits
- Work with a small, senior team advancing AI capabilities across the product.
- Collaborate with product engineers, infrastructure engineers, designers, and product managers.
- Medical, dental, vision, and 401(k) benefits.
- Additional compensation may include equity or commission depending on the position.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →
Похожие вакансии
12 дней назад
Staff Engineer or Architect (Agentic AI)
175 000 - 205 000CAD
10 дней назад
Outcome Engineer (AI)
200 000 - 320 000$
10 дней назад
Outcome Engineer (AI)
200 000 - 320 000$
11 дней назад
Principal Engineer, AI Platform
253 000 - 316 000$
9 дней назад
Outcome Engineer (AI)
200 000 - 320 000$
11 дней назад
AI Engineer
166 000 - 180 000$