Software Engineer, Model Performance Tooling (AI)

Location: Must be based in Vancouver, BC, Canada

Salary: CA$130K - CA$200K

Company

Baseten powers mission-critical inference for the world's most dynamic AI companies by uniting applied AI research, flexible infrastructure, and seamless developer tooling.

What you will do

• Run and automate standard LLM quality benchmarks and custom performance suites for specific workloads.
• Create automated acceptance tests for new GPU clusters, measuring GPU memory bandwidth, networking throughput, and multi-node networking performance.
• Develop and maintain internal GPU-enabled development environments optimized for model experimentation.
• Build and contribute to tools to automate model evaluation and optimization.
• Use PyTorch Profiler and NVIDIA Nsight Systems to collect performance profiles and debug the NVIDIA compute/networking stack.
• Develop real-time dashboards and alerts to monitor system health, model startup times, and runtime performance.

Requirements

• A love for systems & hardware and an understanding of GPU memory subsystems and InfiniBand.
• An automation mindset with a passion for stress-testing and fuzzy testing.
• Mathematical curiosity and a desire to understand the math of Transformers.
• Interest in optimization, including quantization, speculative decoding, and kernel-level optimizations.
• Familiarity with Python and an eagerness to master the NVIDIA software stack.
• C++ familiarity is good to have.

Culture & Benefits

• Competitive compensation, including meaningful equity.
• 100% coverage of medical, dental, and vision insurance for employee and dependents.
• Generous PTO policy including company-wide Winter Break.
• Paid parental leave.
• Company-facilitated 401(k).
• Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.