Эта вакансия в архиве

Посмотреть похожие вакансии ↓
Company hidden
обновлено 1 месяц назад

Software Engineer (Web Crawling)

Формат работы
remote (только USA)
Тип работы
fulltime
Грейд
middle
Английский
b2

Описание вакансии

Текст:
/

TL;DR

Software Engineer (Web Crawling): Develop and optimize web crawlers and scraping infrastructure with an accent on handling anti-bot mechanisms, performance, and security challenges. Focus on designing scalable crawling operations and integrating AI solutions to enhance automation and data extraction accuracy.

What you will do

  • Develop, enhance, and maintain web crawlers and scraping infrastructure.
  • Optimize scraping techniques to handle anti-bot mechanisms, performance, and security challenges.
  • Collaborate with a geographically distributed team to identify and resolve issues.
  • Ensure high availability, efficiency, and reliability of crawling operations.
  • Integrate AI solutions to enhance automation and data extraction accuracy.

Requirements

  • 3+ years of experience in software engineering with a focus on web crawling and data extraction.
  • Strong expertise in Node.js for web crawling applications.
  • Deep understanding of HTML, JavaScript, and reverse engineering techniques.
  • Hands-on experience with Playwright, Puppeteer, and Cheerio for automation and scraping.
  • Knowledge of security and performance best practices related to web crawling.

Nice to have

  • Experience with Apify or Crawlee for large-scale crawling solutions.
  • Proficiency in TypeScript.

Culture & Benefits

  • Unlimited PTO.
  • Comprehensive medical, dental, and vision insurance plans.
  • STD, LTD, AD&D, and life insurance coverage.
  • Free membership to TalkSpace, Teladoc and Health Advocate.
  • 401(k) retirement plan with company matching.
  • Pre-tax commuter benefits and free equipment including laptop and home office stipend.