Software Co-Design AI HPC Systems

Location: Mountain View, United States. Employees living within 50 miles of a U.S. Microsoft office are expected to work from the designated office at least four days per week.

Salary: USD $119,800–$234,700 per year for Software Engineering IC4 roles; USD $142,800–$274,800 per year for IC5 roles. San Francisco Bay Area and New York City metropolitan area ranges are also provided.

Company

Microsoft AI builds AI systems and models intended to advance science, education, productivity, and global well-being.

What you will do

• Lead hardware–software co-design for AI systems spanning accelerators, interconnects, memory, storage, runtimes, and distributed training and inference frameworks.
• Analyze real AI workloads, identify compute, communication, and data-movement bottlenecks, and translate findings into system and hardware requirements.
• Design parallelism strategies, execution models, and distributed algorithms to improve scalability, utilization, reliability, and cost efficiency.
• Develop performance models and capacity plans for future workloads, model architectures, hardware generations, and platform roadmaps.
• Partner with compiler, kernel, runtime, infrastructure, hardware, and product teams to optimize accelerators and productionize co-design ideas.
• Mentor senior engineers and researchers, set technical direction, and contribute to publications, prototypes, or open-source systems where appropriate.

Requirements

• Bachelor’s degree in Computer Science or a related technical field and 6+ years of technical engineering experience, or equivalent experience.
• Preferred: Master’s degree with 8+ years of experience, or Bachelor’s degree with 12+ years of experience.
• Systems-level programming experience with C, C++, CUDA, Python, or comparable languages.
• Experience with AI accelerator or GPU architectures, distributed AI training or inference, HPC, ML systems, runtimes, compilers, or performance modeling.
• Experience designing or operating large-scale AI clusters, accelerator communication stacks such as NCCL, MPI, RDMA, high-speed Ethernet, or InfiniBand, and future hardware capacity planning.
• Experience influencing cross-functional technical decisions and contributing to hardware roadmaps, silicon bring-up, or platform architecture reviews.

Nice to have

• Publications, patents, or open-source contributions in systems, computer architecture, or ML systems.
• Experience with LLMs, multimodal models, or recommendation systems and their systems-level implications.

Culture & Benefits

• Work at the boundary between research exploration and production engineering on datacenter-scale AI platforms.
• Collaborate with internal infrastructure, hardware, compiler, and product teams and external hardware and systems partners.
• Contribute to systems, distributed systems, computer architecture, and high-performance computing communities through research and engineering work.
• Eligible roles may include benefits and additional compensation.
• Employment is subject to Microsoft’s equal opportunity and reasonable accommodation policies.

Hiring process

• Applications are accepted on an ongoing basis until the position is filled, with the posting open for at least five days.