Software Engineer (Data Acquisition)

TL;DR

Software Engineer (Data Acquisition): Develop and maintain scalable distributed systems for data acquisition including web crawling, data ingestion, and search with an accent on backend services, data indexing, and Kubernetes infrastructure. Focus on architecting high-volume data pipelines, ensuring system operability, and collaborating cross-functionally on compliance and performance.

Location: San Francisco, United States (onsite)

Salary: $325,000–$405,000 + Equity

Company

hirify.global is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity by pushing AI capabilities and deploying them safely.

What you will do

• Lead engineering projects in data acquisition including web crawling, data ingestion, and search.
• Collaborate with Data Processing, Architecture, and Scaling teams to ensure smooth data flow and system operability.
• Work with legal team on compliance and data privacy matters.
• Develop and deploy scalable distributed systems handling petabytes of data.
• Architect and implement algorithms for data indexing and search capabilities.
• Build and maintain backend services for data storage and synchronization in Kubernetes environments.

Requirements

• Location: Must be onsite in San Francisco, United States
• BS/MS/PhD in Computer Science or related field.
• 4+ years of industry experience in software development.
• Experience with large stateful distributed systems and data processing.
• Proficiency in Kubernetes and Infrastructure-as-Code concepts.
• Strong communication skills in English.