Senior Technical Program Manager (AI)

TL;DR

Senior Technical Program Manager (AI): Driving the execution of large-scale ML infrastructure and AI tooling initiatives with an accent on model serving, ML pipelines, and real-time inference systems. Focus on bridging the gap between research and production, optimizing system performance, and building internal tools to accelerate model development cycles.

Location: Must be based in the USA

Salary: $152,000 – $190,000 (Base salary only, plus equity and bonus)

Company

Deepgram is a leading Voice AI platform providing real-time APIs for speech-to-text and text-to-speech, powering voice agents for over 1,300 organizations.

What you will do

• Own end-to-end delivery of AI infrastructure programs including training pipelines, experiment tracking, and inference serving.
• Define technical architecture and rollout strategies for new ML systems such as vector databases and model servers.
• Act as the connective tissue between ML research, engineering, product, and data teams to align on roadmaps.
• Drive cost and latency optimization for real-time inference workloads at scale.
• Build lightweight internal tools to accelerate ML iteration cycles and model versioning.
• Identify and resolve technical bottlenecks in training and evaluation workflows.

Requirements

• 5+ years of program management or technical leadership in ML infrastructure or AI tooling.
• Strong technical acumen in ML systems with hands-on experience as an ML or systems engineer.
• Proven ability to translate research requirements into scalable infrastructure.
• Experience coordinating cross-functional ML programs across the full model lifecycle.
• Comfortable navigating complex technical tradeoffs like accuracy versus latency versus cost.
• Familiarity with high-growth or startup environments.

Nice to have

• Hands-on experience with model serving frameworks like vLLM, TensorRT, or TorchServe.
• Experience optimizing LLM or audio model inference (quantization, distillation, batching).
• Knowledge of cost optimization for GPU workloads on cloud or on-premise infrastructure.
• Familiarity with ML experiment tracking tools like MLflow or Weights & Biases.