Назад
Company hidden
23 дня назад

Senior Web Scraper (Scrapy)

50 000 - 65 000€
Формат работы
remote (только Spain)
Тип работы
fulltime
Грейд
senior
Английский
b2
Страна
Spain
Вакансия из списка Hirify.GlobalВакансия из Hirify Global, списка международных tech-компаний
Для мэтча и отклика нужен Plus

Мэтч & Сопровод

Для мэтча с этой вакансией нужен Plus

Описание вакансии

Текст:
/
TL;DR
Senior Web Scraper (Scrapy) (large-scale web scraping): Building distributed scraping architectures and reverse-engineering APIs and mobile apps with an accent on anti-bot evasion, browser automation, and high-volume data extraction. Focus on designing scalable crawlers, detecting scraping blockers, optimizing proxy and session handling, and monitoring failures across thousands of domains.

Location: Remote, Madrid, Spain

Salary: EUR 50,000–65,000 yearly

Company

hirify.global is a consumer intelligence company delivering retail and consumer insights through large-scale data collection, analytics, and digital shelf solutions across more than 100 markets.

What you will do

  • Design and build distributed scraping architectures that scale across thousands of domains.
  • Reverse-engineer APIs and mobile applications using Frida, mitmproxy, Charles Proxy, Burp, and related tools.
  • Detect anti-bot measures, including JavaScript challenges and browser behavior detection, and build human-like adaptive crawlers.
  • Lead scraping innovation by identifying edge cases, prototyping approaches, and sharing knowledge.
  • Set up anomaly detection and monitoring to identify blockers and failures early.
  • Collaborate with Customer Success, Operations, and Data to deliver high-quality actionable data at scale.

Requirements

  • 5+ years of experience with Python and large-scale scraping projects.
  • Deep understanding of anti-bot technologies and bypass methods, including fingerprinting, WAFs, and JavaScript challenges.
  • Experience with Scrapy, Requests, httpx, Selenium, Playwright, or custom headless browser drivers.
  • Experience handling residential, datacenter, and rotating proxies, session management, cookie injection, and TLS tweaking.
  • Strong debugging skills across browsers, HTTP traffic, and device-level interactions.
  • Independent problem-solving skills and willingness to solve novel technical challenges.
  • Familiarity with Parquet, Avro, or real-time stream processing.
  • Knowledge of MongoDB, InfluxDB, and Redis, including optimization.

Nice to have

  • Experience with real-device simulation for mobile scraping.
  • Experience building and maintaining distributed task queues such as Celery or Kafka.
  • Experience scraping high-security platforms or mobile-first environments.
  • Exposure to Linux or cloud environments such as AWS or GCP.
  • Basic understanding of Kubernetes, Cloud Run, and Cloud Functions for deploying and orchestrating scraping workloads.

Culture & Benefits

  • Flexible remote working environment and flexible hours.
  • Ownership and freedom to experiment within an innovation-focused team.
  • Volunteer time off and LinkedIn Learning.
  • Employee Assistance Program and opportunities for professional growth.
  • Competitive salary and collaboration with a global team of scrapers, engineers, analysts, and data specialists.

Будьте осторожны

Если работодатель просит войти в их систему, используя iCloud/Google, прислать код/пароль, запустить код/ПО, не делайте этого. Это мошенники. Обязательно жмите «Пожаловаться» или пишите в поддержку. Подробнее в гайде →