AI Data Engineer

Location: Remote or in-person at the offices in San Francisco

Cash compensation: $150,000–$350,000, including equity incentives

Company

hirify.global is building an open superintelligence stack that combines compute, environments, evaluations, secure sandboxes, training, and deployment for frontier AI research.

What you will do

• Find, evaluate, and collect valuable training data from web, code, and technical sources.
• Build scalable pipelines for extraction, cleaning, deduplication, filtering, and corpus preparation.
• Develop data-quality, coverage, and contamination checks and use them to improve processing.
• Collaborate with researchers on data mixtures, sampling strategies, and experiments for pretraining and midtraining.
• Prepare and evaluate datasets for long-context learning, reasoning, coding, and synthetic data use cases.
• Improve the efficiency, reproducibility, recoverability, provenance, and versioning of large-scale data processing.

Requirements

• Strong Python programming skills and experience building reliable data-processing or distributed systems.
• Experience working with large datasets and balancing storage, compute, throughput, and reproducibility.
• Ability to diagnose data-quality issues and translate research goals into measurable checks and experiments.
• Experience with web crawling, document extraction, code-data collection, corpus curation, deduplication, quality scoring, synthetic data, or contamination analysis.
• Ability to own evolving technical problems end to end and collaborate closely with researchers.

Nice to have

• Experience preparing datasets for language-model pretraining, continued pretraining, or midtraining.
• Experience with Spark, Ray, or comparable distributed processing frameworks.
• Experience with multilingual, mathematical, scientific, or repository-level code data.
• Familiarity with tokenization, training data loaders, and large model-training runs.

Culture & Benefits

• Flexible work arrangements with remote or in-person work options.
• Visa sponsorship and relocation assistance for international candidates.
• Quarterly team off-sites, hackathons, conferences, and learning opportunities.
• Work with a mission-driven team focused on open AI, infrastructure, and accelerating science.