Research Engineer (AI)

TL;DR

Research Engineer (AI): Designing and implementing evaluation infrastructure and post-training pipelines for Kotlin AI coding agents with an accent on agentic error analysis and performance metrics. Focus on building simulation environments, conducting experiments with fine-tuning techniques, and developing standard industry benchmarks for AI-assisted development.

Location: Hybrid roles available in Amsterdam, Belgrade, Berlin, Limassol, Madrid, Munich, Prague, Warsaw, or Yerevan. Relocation support provided to these locations.

hirify.global

hirify.global creates professional developer tools that help software engineers build and deliver high-quality code.

What you will do

• Design and implement tooling to classify and analyze errors generated by AI coding agents.
• Build observability pipelines and simulation environments for benchmarking AI performance.
• Experiment with post-training techniques like SFT, DPO, and GRPO to improve model handling of Kotlin.
• Collaborate with model providers to translate findings into model performance improvements.
• Maintain and evolve open-source benchmarks for measuring AI coding agent performance.
• Own the end-to-end loop of identifying failure modes, designing evals, and shipping fixes.

Requirements

• Strong Python engineering skills (3+ years) with experience in data-heavy codebases.
• Hands-on experience building evaluation or analysis pipelines for LLMs or AI agents.
• Experience with data analysis at scale including SQL and building data pipelines.
• Product-aware mindset with an ability to translate failure modes into actionable research.
• Willingness to develop deep expertise in Kotlin.
• Must be based in or willing to relocate to hirify.global office hubs in the Netherlands, Serbia, Germany, Cyprus, Spain, Czech Republic, Poland, or Armenia.

Nice to have

• Experience with deep learning frameworks like PyTorch and training stacks like TRL or verl.
• Familiarity with agentic frameworks and multi-step coding workflows.
• Knowledge of experiment tracking tools such as Weights & Biases or Langfuse.
• Understanding of the Kotlin ecosystem (Android, Spring, KMP, Gradle).
• Experience contributing to open-source benchmarking projects.

Culture & Benefits

• Flexible work location with options to work from home or the office.
• Remote work policy allowing up to 30 days per year working from abroad.
• Competitive base salary and comprehensive medical insurance allowance.
• Access to conferences, language classes, and continuous learning opportunities.
• Sports benefits and on-site gym access.
• Professional mental health support services.