Member of Technical Staff, Data Infrastructure (AI)

Location: Bay Area, United States; in-office

Annual base salary: $200,000–$350,000 USD, plus equity and benefits.

Company

AI company developing diffusion-based large language models and large-scale AI infrastructure.

What you will do

• Design, build, and operate scalable, fault-tolerant infrastructure for distributed LLM research, including compute, data orchestration, and storage.
• Develop high-throughput data ingestion, processing, and transformation systems for training data catalogs, deduplication, quality checks, and search.
• Build web crawling, data ingestion, and real-time processing systems for model training operations.
• Develop tools for efficient data storage, retrieval, and versioning across distributed systems.
• Work with researchers to accelerate experiments, develop datasets, and improve infrastructure efficiency.
• Ensure data collection follows privacy regulations and ethical AI practices.

Requirements

• BS, MS, PhD, or equivalent experience in Computer Science, Machine Learning, or a related field.
• 3+ years of experience building large-scale data processing pipelines, particularly for AI/ML applications.
• Strong Python skills and experience with Apache Spark, Beam, or Airflow.
• Experience with web scraping, crawling technologies, Common Crawl, SQL, and NoSQL databases.
• Understanding of machine learning fundamentals and experience with PyTorch or TensorFlow.
• Ability to work in the Bay Area in an office-based role.

Nice to have

• Experience with synthetic data generation and data augmentation.
• Experience with LLMs, tokenization, embeddings, and model architectures.
• Experience managing human annotation workflows and quality control.
• Experience with vector databases and embedding-based retrieval.
• Experience with distributed computing and large-scale storage such as HDFS, S3, or BigQuery.

Culture & Benefits

• Collaborate with AI researchers and inventors of diffusion models.
• Competitive salary, equity, and opportunities to shape foundational AI technology.
• Flexible vacation and paid time off.
• Health, dental, and vision insurance, plus 401(k) match.
• Catered meals and commuter subsidies.
• Collaborative and inclusive work environment.