Infrastructure Engineer (AI)

Location: Hybrid, based out of the downtown San Francisco office; remote work is available.

Salary: $200,000–$275,000 per year, depending on experience, plus equity.

Company

LlamaIndex builds AI products and infrastructure for document understanding and knowledge access.

What you will do

• Build and maintain foundational infrastructure that enables engineering teams and supports company growth.
• Design scalable infrastructure for SaaS, single-tenant, and private deployments.
• Manage and optimize cloud resources and Kubernetes clusters for cost, performance, and reliability.
• Improve release and deployment processes and maintain clear infrastructure boundaries for customer deployments.
• Implement security measures and support compliance across deployment environments.
• Build and operate infrastructure for production LLM applications, including model integrations, inference workloads, evaluation pipelines, observability, and agentic workflows.

Requirements

• 8+ years of engineering experience.
• Experience on Platform or Infrastructure teams with Terraform or CDKTF, Kubernetes, Helm, test infrastructure, release management, and observability.
• Experience optimizing cloud resource utilization and tuning Kubernetes clusters for cost and performance.
• Hands-on experience with LLM tooling and production AI systems, including model APIs, agent or RAG frameworks, evaluation and tracing tools, and LLM workload operations.
• Ability to balance speed and pragmatism while building infrastructure for different stages of company growth.
• Willingness to contribute to and help develop the engineering culture.

Nice to have

• Experience building infrastructure from the ground up at a fast-growing startup.
• Experience with Prometheus, Grafana, New Relic, ArgoCD, Flux, SOC2 compliance and audits, Python, Postgres, and multi-cloud deployments.

Culture & Benefits

• Hybrid-friendly work environment with a San Francisco office.
• Opportunity to work on innovative AI products and infrastructure.
• Collaborative engineering culture with opportunities for professional growth.
• Equity compensation and comprehensive medical, dental, and vision coverage.
• Unlimited paid time off, daily catered lunch, and snacks at the San Francisco office.