16 часов назад
AI Agent Architect
Вакансия из Hirify Global, списка международных tech-компанийДля мэтча и отклика нужен Plus
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
AI Agent Architect (LLMs/Agent Evaluation): Building and improving autonomous coding agents that plan, build, test, and deploy production applications with an accent on agent quality, reliability, evaluation, and long-horizon execution. Focus on mining production behavior for hidden failure modes, designing quantitative experiments and evaluation frameworks, and shipping improvements through staged rollouts and rollback criteria.
Location: Bengaluru, India
Company
builds autonomous coding agents that generate, test, and deploy production applications from plain-language intent.
What you will do
- Analyze agent behavior in production to identify failure modes, regressions, and performance bottlenecks.
- Design and run experiments across prompt tuning, evaluation datasets, experimentation systems, and harness engineering.
- Build evaluation frameworks, metrics, datasets, and dashboards for measuring agent quality at scale.
- Establish evaluation gates, staged rollouts, and explicit rollback criteria for production releases.
- Drive improvements in context engineering, memory systems, tool use, and long-horizon execution.
- Make product and engineering decisions in probabilistic systems based on metrics, benchmarks, and observed agent behavior.
Requirements
- 5+ years of experience building and shipping software with end-to-end ownership.
- At least 1 year of hands-on experience with agentic systems and their evaluation.
- Strong systems thinking, product judgment, and independent ownership.
- Practical experience with experimentation, metrics, debugging, and analysis using Python, SQL, or similar tools.
- Ability to reason about noise, confounds, distribution shift, selection effects, and benchmark overfitting.
- Current, hands-on interest in LLMs, AI agents, and emerging model capabilities.
Nice to have
- Experience as a principal or staff individual contributor, software architect, or lead builder.
- Experience moving between engineering and product while continuing to build systems.
- Technical cofounder experience with end-to-end system ownership.
Culture & Benefits
- Ownership of the full improvement loop, from defining experiments to deciding what ships or rolls back.
- Work alongside repeat founders, Olympiad medalists, IIT and IIM alumni, and leaders from Google, Amazon, and Dropbox.
- Daily lunch and dinner provided.
- Family insurance coverage of 3 Lakhs.
- Unlimited paid time off and flexible working hours.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →