AI Infrastructure Engineer (Edge AI)

TL;DR

AI Infrastructure Engineer (Edge AI): Designing, building, and owning the end-to-end infrastructure for serving AI and ML models across edge, cloud, and data center environments with an accent on GPU optimization and model serving. Focus on building fault-tolerant systems, establishing MLOps best practices, and integrating AI inference with power optimization algorithms.

Location: Must be based in the United States

Salary: $170,000 – $210,000 base + stock options

Company

NVIDIA-backed edge AI company enabling greater visibility and control of power utilization in energy-intensive infrastructure like the electric grid and data centers.

What you will do

• Lead the design and build of the AI inference platform, establishing architecture patterns and deployment standards.
• Own end-to-end model serving infrastructure for on-prem and data center environments.
• Build high-performance, fault-tolerant systems for AI model serving focusing on low latency and reliability.
• Optimize GPU utilization and inference performance across the hardware fleet, including NVIDIA accelerators.
• Establish MLOps best practices, including CI/CD pipelines for model deployment, monitoring, and rollback.
• Collaborate with algorithm engineers to integrate AI inference data with power optimization algorithms.

Requirements

• 5+ years of software engineering experience with a focus on AI infrastructure, backend, or distributed systems.
• Hands-on experience with AI model serving frameworks such as vLLM, SGLang, Triton, TensorRT, or TorchServe.
• Proficiency in Python; knowledge of C++, CUDA, Go, or Rust is a plus.
• Understanding of container orchestration and cluster management using Kubernetes and Docker.
• Deep knowledge of GPU workloads and the specific tradeoffs of inference versus training.
• Must be based in the US and willing to travel up to 10% of the time.

Nice to have

• Experience with edge AI deployments or constrained compute environments.
• Familiarity with Infrastructure as Code tools like Terraform and Helm.
• Experience with observability platforms such as Datadog, Prometheus, or Grafana.
• Background in energy, utilities, or industrial IoT.
• Contributions to open-source ML infrastructure projects.

Culture & Benefits

• Competitive compensation including health, dental, and vision insurance.
• Employer-match 401k.
• Flexible work environment with flexible paid time off.
• Mentorship and growth opportunities within a collaborative, lean team.
• Supportive and inclusive workplace culture.