Principal or Staff AI/ML Engineer (AI)

TL;DR

Principal or Staff AI/ML Engineer (AI): Lead development of state-of-the-art inference capabilities on latest hardware for enterprise AI platform with an accent on LLM deployment, lifecycle management, and performance optimization. Focus on mastering new technologies, integrating cutting-edge generative AI serving runtimes, and enabling inferencing on any hardware, cloud, edge, or on-prem.

Location: Fully distributed and remote-first

Company

Microsoft-backed startup developing a software platform for deploying AI inference at scale to any cloud, edge, or on-prem environment.

What you will do

• Partner with customers and hardware teams to solve problems and build inference capabilities.
• Negotiate tradeoffs with product managers and break down features into incremental high-quality deliverables.
• Mentor less experienced team members and propose process improvements.
• Actively participate in discussions, provide proactive communication, and deliver continuous feedback.
• Advocate for perspectives while listening to others in a collaborative team environment.

Requirements

• Excellent verbal and written communication skills
• Expert Python, including extensive packaging experience using Pip, uv, etc.
• Strong experience with vLLM, SGLang, and other modern serving frameworks
• Experience developing high-scale production ML pipelines
• Strong foundation in deep learning architectures, especially Transformer, including NLP tasks like text generation, summarization, and sentiment analysis using LLM APIs/SDK (OpenAI, Anthropic, Mistral, etc.)
• At least eight years related experience

Nice to have

• Experience developing in Rust
• Experience with containers, Docker, and/or Kubernetes
• Experience in AWS, GCP, and Azure
• Experience with AzureML, Google Vertex, Databricks, Sagemaker
• Experience with GPU-focused environments like CUDA, ROCm, or OpenVino
• Prior startup or high-velocity environment experience

Culture & Benefits

• Fully distributed and remote-first team emphasizing self-motivation, independent and collaborative work.
• Thrive in dynamic, fast-paced environments with experimentation, initiative, and adaptability.
• Unlimited time off policy.
• For US employees: Medical, Dental, Vision starting at $1, One Medical, Life insurance, FSAs, Pet insurance, 401K.

Hiring process

• Resume review and initial 45-min screen (technical and behavioral questions, no exercises).
• 90-min technical interview with engineering team (possible take-home).
• Behavioral interview on team fit, communication, decision-making.
• Hiring manager interview on perseverance, conflict resolution, career growth.
• Offer; candidates encouraged to ask questions at each stage.