Senior Data Engineer

TL;DR

Senior Data Engineer: Designing and scaling systems and pipelines that power hirify.global’s data platform with an accent on high-performance, reliable, and maintainable data solutions. Focus on data extraction frameworks, ETL/ELT pipelines, and improving the efficiency, reliability, and performance of data processing systems.

Location: Hybrid (New York)

Salary: $135,000 to $160,000 per year

Company

hirify.global provides a platform that can optimally inform every doctor interaction globally, promoting health equity and building trust in healthcare systems.

What you will do

• Develop strategies and frameworks to capture web data at scale.
• Design, develop, and maintain scalable data extraction frameworks that ingest structured and unstructured data from diverse sources.
• Build and optimize robust ETL/ELT pipelines using big data technologies, especially Apache Spark on cloud platforms (preferably AWS EMR).
• Improve the efficiency, reliability, and performance of data processing systems through thoughtful design and continuous optimization.
• Partner with senior engineers to evolve hirify.global’s data architecture and infrastructure in support of product and platform scalability.
• Collaborate closely with cross-functional teams to align technical execution with business goals.

Requirements

• 5+ years professional experience in data engineering or software engineering, working with large-scale data systems and pipelines.
• Strong proficiency in Python.
• Proficiency in web scraping strategies and technologies: curl, network analysis, proxies and selenium/playwright.
• Strong SQL skills and experience with PostgreSQL.
• Experience with big data tools like Apache Spark, particularly on cloud platforms, with a preference for AWS EMR.
• Experience with Docker or other containerization technologies.

Culture & Benefits

• Full suite of health insurance options, in addition to generous paid time off
• Pre-planned company-wide wellness holidays
• Retirement options
• Health & charitable donation stipends
• Flexible work hours & the opportunity to work from anywhere
• Opportunity to work with leading biotech and life sciences companies in an innovative industry with a mission to improve healthcare around the globe