Staff Software Engineer (GenAI)

Location: San Francisco, California, United States

Salary: $190,900–$232,800 USD per year

Company

A data and AI platform company serving organizations worldwide with products for data engineering, analytics, machine learning, and AI applications.

What you will do

• Own the architecture, design, implementation, and optimization of the GenAI inference engine for large-scale language model serving.
• Partner with researchers to integrate new model architectures and capabilities, including sparsity, activation compression, and mixture-of-experts.
• Optimize latency, throughput, memory efficiency, and hardware utilization across GPUs and accelerators.
• Architect routing, batching, scheduling, memory management, dynamic loading, and distributed inference across nodes.
• Build instrumentation, profiling, and tracing capabilities to identify and resolve performance bottlenecks.
• Ensure reliability, reproducibility, fault tolerance, model versioning, A/B launches, and rollback capabilities while collaborating with platform, infrastructure, and security teams.

Requirements

• BS, MS, or PhD in Computer Science or a related field.
• 6+ years of software engineering experience in performance-critical systems or equivalent experience.
• Experience owning complex system components and making end-to-end architectural decisions.
• Deep understanding of ML inference internals, including attention, MLPs, recurrent modules, quantization, and sparse operations.
• Hands-on experience with CUDA, GPU programming, cuBLAS, cuDNN, and NCCL, plus strong distributed systems knowledge.
• Ability to lead through influence, communicate clearly, and translate research concepts into production systems.

Nice to have

• Published research or open-source contributions in ML systems, inference optimization, or model serving.

Culture & Benefits

• Comprehensive employee benefits and region-specific perks.
• Annual performance bonus and equity may be included in the total compensation package.
• Inclusive hiring practices and a diverse working environment.
• Opportunities to represent the work through benchmarks, whitepapers, and open-source contributions.