System Software Engineer - AI

Location: Hybrid, based in Palo Alto or Charlottesville, Virginia, United States

Salary: $140,000–$200,000 per year, plus equity, benefits, and 401(k).

Company

Stealth-mode startup building foundational technology for performance, scalability, and resiliency in large-scale AI data center clusters.

What you will do

• Design and implement communication and execution primitives connecting high-level AI frameworks with low-level system software.
• Influence the design of foundational infrastructure for next-generation AI models.
• Analyze the software-hardware interface to identify and resolve performance bottlenecks in distributed training and inference.
• Conduct performance benchmarking and characterization on multi-node clusters.

Requirements

• Strong proficiency in C++ and Python.
• Deep understanding of systems programming fundamentals, including memory management, concurrency, and operating system internals.
• Proficiency in a Linux development environment.
• Bachelor’s or Master’s degree in Computer Engineering, Computer Science, or a related field.
• Ability to work in a hybrid role based in Palo Alto or Charlottesville, Virginia.

Nice to have

• GPU programming experience with CUDA and performance optimization for parallel architectures.
• Familiarity with distributed AI frameworks such as PyTorch, JAX, or DeepSpeed.
• Familiarity with inference engines such as vLLM, SGLang, Dynamo, or TRT-LLM.
• Hands-on experience with large-scale cluster orchestration and telemetry tools.

Culture & Benefits

• Work alongside experienced company builders and operators in an early-stage product environment.
• Collaborate across a mix of remote and on-site locations in the United States and Canada.
• Meaningful equity and company benefits.
• 401(k) plan.
• Culture centered on trust, integrity, respect, shared responsibility, and professional growth.