Inference Systems Engineer (AI)

Location: San Francisco, New York City, or Palo Alto, United States

Salary: $220,000–$485,000 per year, plus equity.

Company

Perplexity builds and operates an AI-powered search and answer platform, including the inference infrastructure behind its queries.

What you will do

• Support transformer-based retrieval, text-generation, and multimodal models in the inference infrastructure.
• Develop weight loading, request scheduling, KV-cache management, and API Gateway model support.
• Port CUDA kernels to NVIDIA CuTe DSL for current and future GPU platforms.
• Build and evolve the Rust-native inference serving runtime.
• Profile and optimize performance across network ingress, continuous batching, and GPU kernel interleaving.
• Build dashboards, alerts, and automated remediation while responding to production incidents.

Requirements

• 3+ years of professional software engineering experience with meaningful work in ML inference or high-performance systems.
• Deep experience with GPU programming and performance optimization, such as CUDA, Triton, or CUTLASS.
• Experience operating production distributed systems under real load.
• Working knowledge of Rust, Python, CUDA or CuTe DSL, and at least one deep learning framework such as PyTorch, JAX, or TensorFlow.
• Understanding of GPU architectures, modern LLM architectures, and inference optimization techniques.
• Ability to own problems end-to-end and work independently in a fast-moving environment.

Nice to have

• Experience with PyTorch internals, torch.compile, custom operators, ML compilers, or framework internals.
• Knowledge of NCCL, NVLink, InfiniBand, RDMA, model parallelism, or tensor parallelism.
• Experience with INT8, FP8, or FP4 quantization and mixed-precision serving.
• Familiarity with Nsight, CUDA-GDB, PTX/SASS analysis, Kubernetes, GPU scheduling, or autoscaling inference workloads.

Culture & Benefits

• Full-time U.S. employees receive equity, health, dental, vision, retirement, fitness, commuter, and dependent care benefits.
• Full-time employees outside the U.S. receive benefits tailored to their region of residence.
• The role involves working across research papers, GPU kernels, serving infrastructure, and production incidents.