Staff Site Reliability Engineer (Splunk)

TL;DR

Staff Site Reliability Engineer (Splunk): Owning and evolving the Splunk observability ecosystem with an accent on log management, infrastructure automation, and high-availability systems. Focus on building scalable observability platforms, eliminating toil through automation, and driving observability-driven development across complex distributed systems.

Location: Must be based in the US (Bellevue, Chicago, New York, San Francisco, or Washington, DC). Requires in-person onboarding in San Francisco during the first week.

Salary: $174,000 – $267,000 USD (range varies by location).

Company

Okta is a leading identity and access management provider, securing digital identities for organizations worldwide.

What you will do

• Design, build, and maintain scalable observability infrastructure using Terraform.
• Optimize Splunk log collection, processing, and storage for high reliability and low latency.
• Participate in on-call rotations and lead post-incident reviews to improve system observability.
• Automate the deployment and scaling of observability agents and collectors to eliminate toil.
• Collaborate with SRE teams and business partners to deliver a world-class observability platform.

Requirements

• U.S. Person status required (e.g., U.S. Citizen, Permanent Resident) for federal data access.
• Minimum 5+ years of experience scaling and managing Splunk Cloud at scale (1000+ SVCs).
• Minimum 5+ years of experience in an SRE, DevOps, or Systems Engineering role.
• Strong coding proficiency in Go, Python, or Ruby and expertise in SPL.
• Deep understanding of Linux internals, networking, and Kubernetes/EKS.
• Must be able to attend in-person onboarding in San Francisco.

Nice to have

• Hands-on experience with OpenTelemetry (OTel) or Vector.
• Experience implementing Splunk charge-back applications.
• Experience managing observability tools within AWS or GCP.

Culture & Benefits

• Comprehensive health, dental, and vision insurance.
• 401(k) retirement plan with company matching.
• Flexible spending accounts (FSA).
• Paid leave including PTO and parental leave.
• Global community with a focus on innovation and social impact.