Applied AI Engineer (Operations Automation)

Location: On-site at Bolt's Tallinn HQ, Estonia

Company

Bolt operates a global mobility platform serving more than 200 million customers across 50+ countries and 850+ cities, supported by 4.5+ million partners and 4,000+ employees.

What you will do

• Redesign operational processes by deciding which steps should become software, AI agents, or human checkpoints.
• Build evaluation suites from real cases and human decisions, turning failures into regression tests.
• Develop robust checks for LLM and agent systems, calibrate LLM judges against human labels, and control costs and payouts.
• Run improvement loops across policies, prompts, tools, models, and post-training methods.
• Own rollout decisions, A/B tests, and the feedback loop between live results and evaluation sets.
• Codify successful solutions so other teams can operate them independently.

Requirements

• Background in software engineering, machine learning, or data science, with experience shipping and operating systems that call LLMs.
• Experience building evaluation suites for LLM or agent systems and using them to make release decisions.
• Ability to analyze agent traces, identify decision failures, and improve system behavior.
• Experience designing LLM judges, measuring agreement with human labels, and addressing disagreements.
• Ability to design and interpret A/B tests, compare offline predictions with live results, and assess cost trade-offs between rules, classical ML, humans, and LLMs.
• Clear written English is required; the role is performed on-site in Tallinn, Estonia.

Nice to have

• Hands-on post-training experience with RLVR, GRPO, DPO, reward modeling, or LoRA.
• Experience with GEPA, DSPy, TextGrad, or similar program optimization frameworks.
• RL environment or gym design with programmatic grading.
• Operations automation experience in a marketplace, fintech, or trust and safety team.
• Strong SQL skills and comfort working in unfamiliar codebases.

Culture & Benefits

• On-site collaboration with operations and engineering teams at the Tallinn headquarters.
• Professional growth opportunities and career development.
• Salary, stock options, and physical and mental wellness benefits.
• A one-month paid sabbatical after five years at Bolt.
• Annual company events and smaller team gatherings.