Назад
Company hidden
6 дней назад

Data Annotator (AI)

Формат работы
remote (Global)/onsite
Тип работы
fulltime
Грейд
middle
Английский
b2
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Data Annotator (AI): Annotating multimodal datasets and building high-quality training and evaluation data for machine learning products with an accent on LLM-assisted labeling, annotation quality metrics, and vendor coordination. Focus on designing taxonomies and guidelines, managing dataset lifecycles, implementing human-in-the-loop workflows, and collaborating on model error analysis and active learning.

Location: Fully remote or from a modern office

Company

hirify.global offers the opportunity to work on large-scale products and contribute to their development.

What you will do

  • Annotate text, audio, images, documents, customer interactions, transcripts, and LLM-generated responses using classification, NER, intent extraction, sentiment analysis, and conversation evaluation.
  • Develop annotation guidelines, taxonomies, golden datasets, quality-control tasks, and human-in-the-loop workflows using GPT, Claude, Gemini, and open-source models.
  • Monitor annotation quality through inter-annotator agreement, golden-set accuracy, annotation speed, and cost-efficiency metrics.
  • Coordinate annotators, conduct calibration sessions, provide feedback, and evaluate performance.
  • Manage annotation vendors and platforms, including selection, pilots, SLAs, budgets, deadlines, quality control, and deliverable acceptance.
  • Manage dataset versioning, storage, deduplication, class balancing, data splits, documentation, privacy, and collaboration with Data Science and ML Engineering teams.

Requirements

  • 1.5–2+ years of experience in data annotation or preparing machine-learning datasets, including unstructured text, audio, and image data.
  • Hands-on experience with Label Studio, CVAT, Prodigy, Doccano, or similar annotation platforms, including project setup, templates, and data export.
  • Experience managing crowdsourcing platforms or annotation vendors, including task assignment, quality assurance, deliverable acceptance, and budgets.
  • Experience using LLMs for annotation through web interfaces and APIs, including prompt engineering, few-shot prompting, structured JSON outputs, and quality evaluation.
  • Knowledge of annotation quality and inter-annotator agreement metrics, including kappa, alpha, and precision/recall on golden datasets.
  • Working knowledge of Python for scripting and automation, including Pandas, JSON/CSV/JSONL processing, API calls, and basic data validation, plus SQL for querying, filtering, and aggregating data.
  • Ability to write clear, structured annotation guidelines and technical instructions.
  • English proficiency at B2 level or above for documentation, communication with international vendors, and English-language datasets.

Nice to have

  • Experience preparing datasets for LLM training and evaluation, including SFT conversations, RLHF, preference data, red teaming, and AI assistant response evaluation.
  • Experience in iGaming, betting, e-commerce, FinTech, or customer support data annotation.
  • Speech transcription and annotation experience, including speaker diarization, ASR error analysis, and Whisper.
  • Understanding of fundamental ML concepts, including train/test splits, data leakage, and label noise.
  • Experience with active learning, weak supervision, multilingual datasets, or multilingual annotation teams.

Culture & Benefits

  • Flexible work environment based on autonomy, ownership, and trust without micromanagement.
  • Option to work fully remotely or from a modern office.
  • Medical insurance coverage in countries where the company operates.
  • Annual $2,000 benefits allowance after probation for health insurance upgrades, language learning, and sports activities.
  • Partial reimbursement for professional training, courses, and industry conferences, plus access to a corporate digital library.
  • Company-funded weekly online psychologist sessions and relocation opportunities with assistance after probation, subject to approval.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →