Senior Data Extraction Engineer

TL;DR

Senior Data Extraction Engineer (Web Scraping): Design, develop, and deploy web scraping solutions to collect datasets for AI training with an accent on scalable crawling, data quality, and compliance. Focus on building robust crawlers, preprocessing scraped data for machine learning models, and optimizing crawling performance while collaborating with AI teams.

Location: Chengdu

Company

hirify.global is a global gaming-focused company working across a distributed team.

What you will do

• Design, develop, and deploy web scraping solutions to collect specific datasets for AI training.
• Build robust, scalable web crawlers to extract structured and unstructured data from online sources.
• Ensure data accuracy, integrity, and compliance with applicable laws and regulations.
• Clean, preprocess, and organize scraped data for use in machine learning models.
• Monitor and optimize crawling performance for efficiency and reliability.
• Collaborate with AI teams to define data requirements and document crawling workflows, tools, and results.

Requirements

• Bachelor’s or master’s degree in computer science, software engineering, or a related field.
• Strong experience with web scraping tools and frameworks (Scrapy, Selenium, BeautifulSoup).
• Proficiency in Python, Java, or Node.js.
• Familiarity with HTTP protocols, HTML parsing, and JSON data formats.
• Knowledge of database systems (SQL, NoSQL) for data storage and management.
• Experience with cloud platforms (AWS, GCP) and containerization (Docker).

Nice to have

• Experience with large-scale data scraping and distributed crawlers.
• Familiarity with AI and machine learning concepts, especially data preprocessing.
• Knowledge of browser automation for dynamic content rendering.
• Ability to handle multilingual data and diverse data formats.

Culture & Benefits

• Work with a global team across multiple continents.
• Gamer-centric culture and an emphasis on accelerated personal and professional growth.
• Opportunity to make an impact on a global mission.