2 дня назад
Backend Engineer (AI)
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Описание вакансии
Текст:
TL;DR
Backend Engineer (AI): Building and operating the evaluation framework for computer-use agents across web apps, desktop applications, and command-line environments with an accent on orchestration, reliability, observability, and reproducible results. Focus on scaling benchmark execution, integrating new evaluations, optimizing Kubernetes workloads, and turning customer workflows into automated measurements for research and product releases.
Location: Hybrid in London or Paris. The London posting requires three office days per week and a trip to Paris approximately every 4–6 weeks.
Company
H builds computer-use agents and the models behind them, providing a managed API and deploying agents into enterprise workflows.
What you will do
- Build and operate the evaluation framework for benchmarks across web applications, desktop software, and command-line environments.
- Integrate benchmarks for researchers and forward deployed engineers while shortening the integration path for future evaluations.
- Define benchmark standards and implement checks that enforce reliable, reproducible results.
- Improve scheduling, cluster utilization, observability, and evaluation performance across approximately 50 benchmarks, growing toward 100–200.
- Work with customers to identify workflows they want measured and automate the flow of results into evaluation harnesses and models.
Requirements
- 5+ years of backend development experience with production Python.
- Experience building test, QA, or evaluation tooling used by other engineering teams.
- Experience operating distributed systems on Kubernetes in a public cloud; AWS experience is valuable.
- Experience delivering systems end to end, including REST or GraphQL APIs and third-party integrations.
- Knowledge of relational and non-relational databases and message queues such as SQS, RabbitMQ, or Kafka.
- Experience implementing metrics, tracing, and monitoring from the start of a project.
Nice to have
- Experience measuring LLM quality or building agents.
- Experience with Docker, virtual machines, Temporal, Dask, FastAPI, PostgreSQL, Grafana, or Datadog.
- Experience automating web or desktop software with Playwright, Selenium, or browser extensions.
- Experience setting engineering standards through code reviews, design reviews, or mentoring.
Culture & Benefits
- Full-time hybrid work in London or Paris.
- Close collaboration with researchers and forward deployed engineers.
- The role offers ownership of areas such as run scaling, observability, or related benchmark groups.
- Competitive package.
Hiring process
- 30-minute call with the Talent team.
- 60-minute technical challenge and 60-minute system design interview.
- 30-minute final conversation with the VP of Engineering, approximately 3.5 hours in total.
Будьте осторожны: если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого - это мошенники. Обязательно жмите "Пожаловаться" или пишите в поддержку. Подробнее в гайде →