Research Engineer (AI Agents)

Location: On-site in New York, NY or San Francisco, CA

Company

LangChain builds open-source frameworks and the LangSmith platform for building, evaluating, deploying, and operating AI agents at scale.

What you will do

• Build and maintain benchmarks and evaluations that measure the quality and efficiency of LangSmith Engine agents on real-world tasks.
• Design and run experiments across models, prompting, context, tools, orchestration, and agent strategies.
• Explore post-training and fine-tuning techniques to improve agent capabilities, quality, and cost.
• Turn successful research experiments into production improvements and prevent regressions.
• Study real agent failures, define the ML roadmap, and improve agent performance in collaboration with production engineers and researchers.
• Mentor engineers through strong technical leadership and clear communication of research findings.

Requirements

• 4+ years of experience in ML/AI research or a closely related field.
• Master’s or Ph.D. in a relevant scientific field; a Ph.D. in Machine Learning, Computer Science, or Physics is valued.
• Hands-on experience with LLMs and AI agents, including analyzing model behavior and improving real-world performance.
• Strong experience designing benchmarks, evaluations, and experiments for AI/ML systems.
• Strong software engineering skills with experience taking research prototypes to measurable production impact.
• Strong research judgment, autonomy, ability to work through ambiguity, and clear technical communication.

Nice to have

• Experience with LLM-as-a-judge, automated graders, synthetic data generation, or human evaluation.
• Experience with reinforcement learning, preference optimization, SFT, RLHF/RLAIF, or other LLM post-training techniques.
• Experience optimizing LLM agents for production cost, latency, or task efficiency.
• Experience with model serving, inference optimization, distributed systems, or GPU infrastructure.

Culture & Benefits

• Opportunity to influence the development of agent engineering technology and production AI systems.
• Competitive base compensation, variable compensation where relevant, meaningful equity, benefits, and perks.
• Medical, dental, and vision coverage, flexible vacation, a 401(k) plan, and life insurance.
• Meals on in-office days in the US.