Staff Site Reliability Engineer (Splunk)

Location: Hybrid role in Bellevue, Chicago, New York, San Francisco, or Washington, DC; in-person onboarding is required in the San Francisco or Chicago office during the first week.

Salary: $174,000–$239,000 USD annual base salary; location-specific ranges may reach $267,000.

Company

Okta builds identity and security infrastructure for organizations and AI systems.

What you will do

• Design, build, and maintain scalable observability infrastructure with Terraform.
• Own and evolve the Splunk ecosystem, optimizing log collection, processing, storage, reliability, and latency.
• Automate deployment and scaling of observability agents and collectors across distributed systems.
• Participate in on-call rotations and lead post-incident reviews to drive systemic improvements.
• Build internal tools and workflows using SPL and Go.

Requirements

• U.S. Person status is required to access federal environments or protected federal data.
• At least 5 years of experience scaling and managing Splunk Cloud, including environments with 1,000+ services, Workload Management, and HEC optimization.
• At least 5 years of experience in SRE, DevOps, or systems engineering focused on highly available systems.
• Strong proficiency in SPL and Go, plus knowledge of Linux internals, networking, TCP/IP, DNS, load balancing, and Kubernetes/EKS.
• Ability to create actionable Splunk dashboards and debug complex cross-service performance bottlenecks.

Nice to have

• Experience with OpenTelemetry, Vector, or similar telemetry frameworks.
• Experience implementing Splunk charge-back applications for usage reporting.
• Experience managing native observability tools in AWS or GCP.

Culture & Benefits

• Health, dental, and vision insurance.
• 401(k) and flexible spending account.
• Paid leave, including PTO and parental leave.
• Equity and bonus opportunities where applicable.
• In-person onboarding designed to connect new hires with the mission and team.