Principal AI Product Engineer (AI)

Location: Houston, New York, San Francisco, or Seattle, United States

Salary: $290,000–$520,000 USD per year, plus potential bonus, equity, and/or commission.

Company

Nscale is building a vertically integrated GenAI cloud platform spanning data centers, software, and AI applications.

What you will do

• Define the multi-year technical roadmap for inference, evaluation, and post-training platforms.
• Lead architecture for serving, KV-cache orchestration, speculative decoding, multi-tenant scheduling, quantization, sparsity, MoE serving, and RL infrastructure.
• Set engineering standards for APIs, compatibility, benchmarking, evaluation methodology, training stability, and performance testing.
• Develop frameworks for balancing cost, latency, throughput, and model quality.
• Align AI engineering, research, product, and infrastructure leaders on technical strategy and commercial trade-offs.
• Mentor Staff and Senior AI Engineers and represent Nscale through open source, publications, conferences, and industry partnerships.

Requirements

• 10–15 years of engineering experience with pillar-level impact on production AI systems.
• 4+ years of hands-on experience with LLM inference, GPU performance, evaluations, post-training, or reinforcement learning.
• Experience defining multi-year strategy and owning architecture for large-scale production inference or training platforms.
• Deep knowledge of AI accelerator hardware and software, including CUDA or ROCm, memory bandwidth, interconnects, and distributed computing.
• Experience creating engineering standards adopted across large organizations and growing Staff-level technical leaders.
• External recognition through AI systems research, open source, or industry contributions.

Nice to have

• Experience at a leading AI lab or hyperscaler AI infrastructure team.
• Contributions to inference, kernel, or RL frameworks such as vLLM, SGLang, TensorRT-LLM, Triton, verl, DeepSpeed, or Megatron-LM.
• Hands-on experience with DPO/GRPO-style methods, reward modeling, multi-turn and tool-use RL, and inference-training integration.
• Experience with developer API platforms, control plane/data plane architecture, cell-based deployments, or hardware-software co-design.
• Published work in AI systems or experience defining pricing, SLO, and capacity models for commercial inference products.

Culture & Benefits

• Fast-paced environment focused on shipping, measuring, and iterating.
• End-to-end ownership, accountability, adaptability, and collaborative execution.
• Medical, dental, and vision coverage.
• Flexible paid time off, parental leave, and retirement plan participation.