Software Engineer, Ml & Data Infra (AI)

TL;DR

Software Engineer, ML & Data Infra (AI): Building foundational infrastructure for frontier AI models, focusing on petabyte-to-exabyte scale distributed systems for data acquisition, web crawling, and multimodal pipelines. Focus on high-performance search/retrieval engines and low-level performance optimization using CUDA kernels and compiler/runtime innovations.

Location: Must be based in Palo Alto, CA

Salary: $180,000 - $440,000 USD

Company

hirify.global’s mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge.

What you will do

• Design, build, and operate petabyte-to-exabyte scale distributed systems for data acquisition, web crawling, preprocessing, filtering/classification, and multimodal pipelines.
• Architect high-performance search/retrieval engines at trillion-document scale, integrating with LLMs/agents for truth-seeking and real-time knowledge access.
• Develop reliable inference serving infrastructure, including load balancing, autoscaling, and monitoring for 100% uptime and optimal tail latency.
• Optimize low-level performance using CUDA kernels, Triton/CUTLASS extensions, and model-hardware co-design.
• Innovate on compilers/runtimes, distributed profiling/debugging tools, and interconnect fabrics.
• Manage complex workloads across clouds/clusters, including orchestration, data bookkeeping, and failure analysis.

Requirements

• Strong systems engineering skills with proven impact on large-scale distributed infrastructure.
• Proficiency in Python and at least one compiled language (Rust, C++, Go, Java).
• Hands-on experience with at least one key area: data pipelines/crawling, web-scale search/retrieval, inference optimization, compiler features, or high-speed interconnects.
• Deep understanding of distributed systems challenges, including high-throughput ops/sec, latency/throughput tradeoffs, and fault-tolerance.
• Passion for AI infrastructure and delivering rigorous, high-quality results.

Nice to have

• Experience with multimodal data, epistemics/truth-seeking in retrieval, or agentic systems.
• Low-level optimizations experience, including CUDA kernel development and GPU profiling.
• Production expertise in inference reliability, CI/CD for ML, or cluster networking.
• Track record owning end-to-end projects in hyperscale environments.

Culture & Benefits

• Equity, comprehensive medical, vision, and dental coverage.
• Access to a 401(k) retirement plan.
• Short & long-term disability insurance.
• Life insurance.
• Various other discounts and perks.