Назад
Company hidden
4 дня назад

Platform Engineer (RL & Eval Data)

Формат работы
remote (только Argentina)
Тип работы
fulltime
Грейд
senior
Английский
c1
Страна
Argentina
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Platform Engineer (RL & Eval Data) (Python, Kubernetes, LLM Evaluation): Maintaining and extending evaluation, reinforcement learning, and annotation platforms with an accent on scalable backend systems, sandbox infrastructure, and reliable production operations. Focus on building RL environments, orchestrating high-throughput rollouts, securing untrusted code execution, and creating evaluation and human-data tooling.

Location: Argentina — fully remote

Company

hirify.global's Human Data Division builds training and evaluation data for frontier AI labs, including RL environments, coding and STEM evaluation tasks, and expert-annotated datasets.

What you will do

  • Maintain and improve evaluation, RL, and annotation platforms in production through bug fixing, technical-debt reduction, testing, and documentation.
  • Design and ship features across databases, backend services, APIs, and internal tools.
  • Scale distributed systems for larger workloads with predictable performance and cost.
  • Build RL environments and infrastructure for sandboxes, environment images, rollouts, and reward computation.
  • Improve reliability and security through monitoring, alerting, incident response, access control, and safe handling of untrusted code.
  • Build tooling for task authors, reviewers, and project leads, including fast-turnaround samples.

Requirements

  • 5+ years in software, data, or ML engineering, including recent work in AI/ML data, model evaluation, annotation, or developer tooling.
  • Strong backend engineering in Python with clean, tested, maintainable code; experience joining and improving existing codebases.
  • Experience designing APIs, data models, relational database schemas, migrations, and query performance optimization with Postgres or similar systems.
  • Enough TypeScript and React experience to build and maintain internal tools.
  • Experience designing distributed systems with long-running concurrent jobs, orchestration, workflow engines, queues, retries, idempotency, and failure recovery.
  • Experience with containers, Docker, Kubernetes or similar orchestration, and at least one major cloud provider; knowledge of logging, metrics, tracing, alerting, secrets management, isolation, and access control.
  • Hands-on experience building RL environments for LLM agents, including tool interfaces, seedable/resettable/snapshotable state, and reward computation.
  • Experience running agent code in isolated sandboxes at scale, building and caching reproducible environment images, collecting high-throughput rollouts, and feeding rollouts and rewards into RL post-training pipelines.
  • Hands-on experience with LLM evaluation harnesses or benchmarks, verifiers, graders, reward functions, frontier-model evaluation concepts, provider APIs, and model cost and usage tracking.
  • Experience building annotation, labeling, or human-data tooling; contributor and reviewer workflows; quality control and calibration; and scaling manual processes.
  • Daily, careful use of AI coding agents, independent execution in ambiguous environments, initiative, and clear written communication with technical and non-technical colleagues.
  • Fluent English required.
  • A track record of running and maintaining production systems, not only building them.

Nice to have

  • Contributions to open-source evaluation or RL frameworks.
  • Experience building tools for a large distributed contributor workforce.

Culture & Benefits

  • Full-time, fully remote work from Argentina.
  • Small, fast-moving team with a high quality bar.
  • Work closely with task authors, data reviewers, project leads, and AI labs.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →