Вакансия из Hirify Global, списка международных tech-компаний Для мэтча и отклика нужен Plus
Пожаловаться
Зарплата и рынок
развернуть
По международному рынку данных пока нет
На СНГ рынке: $3.7к/мес ($3к - $4.3к)
89
Крутая вакансия
развернуть
Чётко описанная senior-роль в AI-продукте с серьёзной ответственностью за сбор данных и production-инфраструктуру; главный компромисс - широкий и требовательный объём задач.
?Широкая зона ответственности
Современный стек
Сфера AI
Понятные метрики
Ответственный сбор данных
Мэтч & Сопровод
Для мэтча с этой вакансией нужен Plus
Подключить Plus
Описание вакансии
Текст:
Hirify AI/Оригинал
TL;DR
Member of Technical Staff (AI Knowledge Platform): Building acquisition systems, event-driven ingestion, and lakehouse pipelines that transform web captures and large relational datasets into reproducible data for AI research, training, and production predictions with an accent on scraping reliability, streaming, data quality, and schema evolution. Focus on designing resilient crawlers and browser automation, operating Kafka and Pub/Sub pipelines, and ensuring traceable, versioned datasets at scale.
Location: Remote, São Paulo
Company
hirify.global is a frontier AI lab building relational foundation models from the ground up for enterprise decision-making in credit, growth, and fraud across Latin America.
What you will do
Build and operate crawlers, scrapers, and browser RPA with scheduling, retries, session handling, change detection, coverage, and freshness monitoring.
Run event-driven ingestion with Kafka and Pub/Sub, including ordering, replays, dead-letter queues, backpressure, and idempotent consumers.
Build batch and incremental Spark, dbt, and SQL pipelines that transform raw captures into clean, typed relational tables.
Own Parquet and Iceberg lakehouse systems, including schema evolution, partitioning, compaction, and reproducible snapshots.
Establish data quality, lineage, and source traceability standards, and operate systems through incidents and source changes.
Deploy and manage infrastructure on Kubernetes with Helm, Terraform, and CI/CD pipelines with reliable rollback, while collecting data responsibly with Security and Legal.
Requirements
5+ years of relevant engineering experience at technology and product-driven companies.
Production experience in at least one of web scraping, large-scale crawlers, or browser RPA, including collection from real sources rather than only consuming APIs.
Data engineering experience with large data volumes using Spark, dbt, and SQL, plus columnar and table formats such as Parquet and Iceberg.
Production experience with Kafka and/or Pub/Sub.
Production coding experience in Python and JavaScript for pipelines, crawlers, and browser automation.
Hands-on Kubernetes experience, including Helm packaging.
Experience with Terraform, CI/CD pipelines, and reliable rollback.
Willingness to operate built systems and respond to incidents when sources or pipelines fail.
Daily use of coding agents and a considered approach to using them safely.
Nice to have
Experience with difficult sources involving heavy JavaScript rendering, logged-in sessions, rate limits, proxy management, Playwright, Puppeteer, or Scrapy.
Airflow, Dagster, or Temporal workflow orchestration.
Entity resolution and relational or graph data modeling across many sources.
ML datasets involving point-in-time correctness, reproducible training sets, and evaluation splits.
GCP and AWS data services, including GKE, BigQuery, Dataproc, and EKS.
OpenTelemetry observability.
Financial services or regulated-environment experience, including LGPD and ISO 27001.
Culture & Benefits
Remote work from São Paulo.
Hands-on builder environment focused on operating production systems.
Responsible data collection in partnership with Security and Legal, including rate limits, terms of use, LGPD, and personal-data handling.
Success is measured through freshness, source coverage, recovery time, pipeline reliability, data quality, and processing cost.
Будьте осторожны
Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →