Вакансия из Hirify Global, списка международных tech-компаний Для мэтча и отклика нужен Plus
Пожаловаться
77
Хорошая вакансия
развернуть
Роль описана чётко и сочетает работу с AI-данными с управлением подрядчиками и качеством; главный вопрос - в вакансии мало информации о компании и продукте.
?Много направлений
Работа с AI
Поддержка обучения
Мало о компании
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Подключить Plus
Описание вакансии
Текст:
Hirify AI/Оригинал
TL;DR
Data Annotator (AI): Annotating multimodal datasets and building high-quality training and evaluation data for machine learning products with an accent on LLM-assisted labeling, annotation quality metrics, and vendor coordination. Focus on designing taxonomies and guidelines, managing dataset lifecycles, implementing human-in-the-loop workflows, and collaborating on model error analysis and active learning.
Location: Fully remote or from a modern office
Company
hirify.global offers the opportunity to work on large-scale products and contribute to their development.
What you will do
Annotate text, audio, images, documents, customer interactions, transcripts, and LLM-generated responses using classification, NER, intent extraction, sentiment analysis, and conversation evaluation.
Develop annotation guidelines, taxonomies, golden datasets, quality-control tasks, and human-in-the-loop workflows using GPT, Claude, Gemini, and open-source models.
Monitor annotation quality through inter-annotator agreement, golden-set accuracy, annotation speed, and cost-efficiency metrics.
Coordinate annotators, conduct calibration sessions, provide feedback, and evaluate performance.
Manage annotation vendors and platforms, including selection, pilots, SLAs, budgets, deadlines, quality control, and deliverable acceptance.
Manage dataset versioning, storage, deduplication, class balancing, data splits, documentation, privacy, and collaboration with Data Science and ML Engineering teams.
Requirements
1.5–2+ years of experience in data annotation or preparing machine-learning datasets, including unstructured text, audio, and image data.
Hands-on experience with Label Studio, CVAT, Prodigy, Doccano, or similar annotation platforms, including project setup, templates, and data export.
Experience managing crowdsourcing platforms or annotation vendors, including task assignment, quality assurance, deliverable acceptance, and budgets.
Experience using LLMs for annotation through web interfaces and APIs, including prompt engineering, few-shot prompting, structured JSON outputs, and quality evaluation.
Knowledge of annotation quality and inter-annotator agreement metrics, including kappa, alpha, and precision/recall on golden datasets.
Working knowledge of Python for scripting and automation, including Pandas, JSON/CSV/JSONL processing, API calls, and basic data validation, plus SQL for querying, filtering, and aggregating data.
Ability to write clear, structured annotation guidelines and technical instructions.
English proficiency at B2 level or above for documentation, communication with international vendors, and English-language datasets.
Nice to have
Experience preparing datasets for LLM training and evaluation, including SFT conversations, RLHF, preference data, red teaming, and AI assistant response evaluation.
Experience in iGaming, betting, e-commerce, FinTech, or customer support data annotation.
Speech transcription and annotation experience, including speaker diarization, ASR error analysis, and Whisper.
Understanding of fundamental ML concepts, including train/test splits, data leakage, and label noise.
Experience with active learning, weak supervision, multilingual datasets, or multilingual annotation teams.
Culture & Benefits
Flexible work environment based on autonomy, ownership, and trust without micromanagement.
Option to work fully remotely or from a modern office.
Medical insurance coverage in countries where the company operates.
Annual $2,000 benefits allowance after probation for health insurance upgrades, language learning, and sports activities.
Partial reimbursement for professional training, courses, and industry conferences, plus access to a corporate digital library.
Company-funded weekly online psychologist sessions and relocation opportunities with assistance after probation, subject to approval.
Будьте осторожны
Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →