Senior Python Developer (Scrapy)

TL;DR

Senior Python Developer (Scrapy): Building and optimizing robust web scraping solutions for large-scale data collection with an accent on anti-bot countermeasures and system resilience. Focus on architecting scalable crawling infrastructure, ensuring high data quality, and mentoring the team on advanced scraping techniques.

Location: Lisbon, Portugal

Compensation: EUR 38,400 – EUR 54,700 yearly

Company

hirify.global is a global consumer intelligence company providing comprehensive insights into buying behavior through advanced analytics and state-of-the-art platforms.

What you will do

• Design and implement robust Scrapy spiders capable of bypassing advanced anti-bot countermeasures.
• Conduct code reviews and data validation to ensure high standards of quality and reliability.
• Architect new developments across multiple codebases and provide technical documentation.
• Leverage deep knowledge of HTTP protocols and modern web browser mechanisms for efficient crawling.
• Conduct training sessions for relevant teams to share technical expertise.

Requirements

• Master's degree in Computer Science, IT, or a related field.
• At least 5-6 years of professional experience in software engineering.
• Proficiency with Scrapy or BeautifulSoup frameworks.
• Solid understanding of HTML, JavaScript, and single-page applications.
• Strong skills in Docker, Git, pandas, Linux, and bash scripting.
• Experience with major cloud providers such as AWS, GCP, or Azure.

Nice to have

• Experience with time series databases like InfluxDB.
• Hands-on experience with Python web frameworks such as Django or FastAPI.
• Familiarity with RESTful and GraphQL APIs.

Culture & Benefits

• Flexible working environment.
• Access to LinkedIn Learning for professional development.
• Employee-Assistance-Program (EAP) for personal support.
• Volunteer time off opportunities.